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Section  1  Summary 


The  influence  of  parallel  scanned  focal  plane  array  (FPA)  design  parameters  on  future  automatic 
taiget  recognizer  (ATR)  performance  was  examined.  The  design  variables  tested  were 
instantaneous  field  of  view  (IFOV)  size  and  shape,  sampling  rate,  sample-scene  phasing  effects, 
signal  to  noise  ratio  (SNR),  residual  detector-to-detector  nonuniformity  and  interlace  effects. 
Simulated  forward  looking  infrared  system  (FLIR)  designs  for  this  experiment  were  chosen  to 
span  design  choices  similar  to  typical  “first”  and  “second  generation”  tactical  Army  FLIRs. 

The  influence  of  field  misregistration  due  to  the  interaction  of  jitter  and  interlaced  scanning  was 
investigated  first  using  a  statistical  classifier  as  the  ATR.  Misregistration  of  fields  by  one  or  two 
pixels  was  shown  to  degrade  classification  perfonnance  significantly.  The  influence  of  interlace 
on  three  different  types  of  trackers  was  also  examined.  Interlace  was  shown  to  either  seriously 
degrade  tracker  performance  or  cause  the  tracker  to  completely  break  lock- 

in  an  effort  to  make  the  results  of  the  testing  applicable  to  a  large  class  of  algorithms  and  not  be 
algorithm  specific  a  more  general  approach  was  taken.  A  high  resolution  reference  image  was 
correlated  with  the  same  image  after  having  been  passed  through  a  simulated  atmosphere  and 
FLIR.  The  degraded  image  was  systematically  shifted  with  respect  to  the  starting  image  and  a 
correlation  surface  was  calculated,  the  peak  of  which  was  used  as  a  measure  of  future  algorithm 
performance. 

In  the  absence  of  atmospheric  effects,  the  noise  values  chosen  for  this  experiment  did  not  degrade 
correlation  significantly  since  the  signal  from  target  and  background  was  so  large  relative  to  the 
noise.  Almost  all  correlation  degradation  with  range  was  caused  by  blurring  due  to  optics, 
detector  and  the  sampling  process. 

The  simulated  atmospheres  by  themselves,  while  seriously  degrading  the  contrast  in  the  image, 
had  very  little  effect  on  correlation  peak  values.  However,  when  noise  and  atmospheric  effects 
were  simulated  together,  they  drastically  degraded  the  signal  to  noise  in  the  image,  and  the 
correlation  peak  values  were  significantly  reduced  to  the  point  where  the  image  would  not  be 
usable  for  automatic  target  recognitioa 

Of  the  three  values  for  nonunifonnity  examined,  only  the  largest  was  seen  to  seriously  degrade 
correlation  when  examined  alone.  As  with  the  noise  case,  combining  the  influence  of 
nonuniformity  and  atmosphere  drastically  degraded  correlation  values. 

Increasing  the  sampling  rates  while  holding  other  variables  constant  improved  performance,  as 
would  be  expected.  The  correlation  peak  value  for  1.5  samples  per  dwell  (spd)  was  equivalent  to 
the  correlation  peak  for  a  similar  system  with  1.0  spd  at  KXX)  meters  shorter  range.  That  is,  the 
same  quality  image  was  obtained  at  greater  range  with  the  higher  sampling  rate.  Similarly,  the 
correlation  values  for  2.0  spd  were  equal  to  those  for  1.5  spd  at  approximately  500  meters  closer 
range.  These  trends  held  for  all  ranges  tested  (3  -  6  km)  and  for  both  square  and  rectangular 
detectors. 
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Correlation  was  shown  to  be  strongly  influenced  by  the  number  of  samples  on  target  up  to 
approximately  500  samples  /  mrad2.  This  is  consistent  with  past  observations  that  algorithm 
performance  is  a  strong  function  of  the  number  of  pixels  on  target 

Larger  detectors  degraded  the  system  resolution  and  thus  the  correlation  values.  However,  if  the 
system  was  assumed  to  be  BLIP  limited  with  larger  detectors  resulting  in  less  noise  in  the  final 
image,  the  larger  detectors  were  shown  to  have  advantages  in  low  SNR  conditions. 

Sample-scene  phasing  causes  variation  of  the  target  appearance  for  slightly  different  alignments 
of  a  FLIR  line  of  sight  (LOS).  The  effect  is  most  pronounced  for  the  lowest  sample  rate  tested,  1 
spd,  and  is  most  prominent  when  the  image  is  improperly  reconstructed.  The  effect  is  strongly 
range  dependent,  being  most  noticeable  at  longer  ranges.  For  2  spd  and  good  reconstruction  the 
effect  is  negligible,  even  at  long  ranges,  as  would  be  expected.  Minimizing  the  variation  of  the 
appearance  of  the  target  with  changing  LOS  will  decrease  the  frame  to  frame  variation  in 
algorithm  performance,  thus  increasing  robustness.  Thrs  robusmess  afforded  by  the  high 
sampling  rates,  especially  at  the  longer  ranges  is  a  good  reason  to  push  FPA  designs  toward 
higher  sampling  rates.  This  phasing  effect  has  not  been  explored  in  previous  efforts. 


Preliminary  investigations  of  image  restoration  show  promise.  We  will  continue  to  investigate  the 
benefits  of  restoration  of  simulated  and  real  FLIR  imagery. 
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Section  2  Introduction 


The  overall  perfoiroance  of  a  thermal  imager,  whether  designed  to  produce  an  image  for  a  human 
or  an  autoprocessor,  depends  critically  upon  the  design  and  performance  of  the  focal  plane  array 
(FPA).  As  the  U.  S.  begins  the  design  and  constmction  of  a  second  generation  of  infrared  FPAs  it 
is  important  to  investigate  the  influence  of  various  array  designs,  not  only  on  the  human,  the 
traditional  end  user  of  the  IR  image,  but,  equally  as  important,  on  autoprocessors  which  are  taken 
here  to  include  trackers,  autocuers  and  automatic  target  recognizers. 

First  generation  FLIR  systems  currently  in  the  Army  inventory  produce  imagery  with  numerous 
artifacts  such  as  streaking  caused  by  AC  coupling,  and  jagged  edges  caused  by  imperfect 
interlace.  The  artifacts  often  produce  dramatic  variations  in  the  appearance  of  a  target  from 
frame  to  fiame.  To  a  human  observer  whose  eyes  integrate  a  displayed  image  for  approximately 
0.2  seconds  (or  about  6  frames)  such  variation  may  be  noticed  only  as  a  slight  blurring  of  the 
imagery.  However,  to  a  target  recognizer  which  operates  on  every  frame  of  imagery,  the  frame  to 
frame  variation  can  be  disastrous.  Figure  1  shows  the  result  of  a  statistical  classifier  operating  on 
a  consecutive  sequence  of  60  first  generation  FLIR  images  of  a  stationary  target.  The  target  has 
been  segmented  in  each  image  by  the  algorithm  and  the  target’s  height-to-width  ratio  has  been 
measured.  This  ratio  is  one  of  several  which  are  used  by  the  algorithm  to  discriminate  between 
different  target  types.  It  is  easy  to  see  how  the  frame  to  frame  instability  of  the  first  generation 
imager  could  cause  drastic  variations  in  algorithm  performance. 


Figure  1.  Frame  to  fk'ame  variations  of  the  (height/width)^  ratio 
for  60  consecutive  frames  of  first  generation  FLIR  imagery 


The  FPA  candidates  which  were  thought  to  best  address  U.  S.  Army  weapon  system  applications 
were  parallel  scanning  FPAs  of  different  lengths  and  architectures.  Critical  FPA  design  issues 
involved  parameters  which  not  only  strongly  impacted  system  performance  but,  at  the  same  time, 
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directly  influenced  FPA  producibility  and  cost  Those  FPA  design  parameters  which  were 
identified  as  being  crucial  to  both  system  performance  and  FPA  producibility  were  the  following. 

1 .  Interlaced  vs.  noninterlaced  image  format. 

2.  Sample  spacing  in  both  scan  and  orthoscan  directions. 

3.  Signal  to  noise  (SNR),  including  nonunifoimity. 

4.  Detector  size  and  geometry  (rectangular  vs.  square). 

The  goal  of  this  study  was  to  quantify  the  effects  of  these  parameters  on  algorithm  p)erformance 
separately  so  that  cost/performance  trade-offs  could  be  made.  Ideally  this  study  should  be 
designed  to  determine  the  effects  of  these  parameters  on  the  next  generation  of  autoprocessors. 
The  focus  of  current  algorithm  research  is  shifting  ftom  statistical  classifiers  to  template 
matching  or  model  based  algorithms.  These  future  algorithms  could  have  totaUy  different 
sensitivities  to  FPA  designs  than  statistical  classifiers,  and  we  should  not  be  designing 
tomorrow’s  FPA  for  use  with  yesterday’s  algorithm.  Yet,  without  tomorrow’s  algorithms  in  hand, 
it  is  difficult  to  perform  testing  which  properly  indicates  the  sensitivities  of  future  algorithms  to 
various  FPA  designs. 

This  study  began  with  an  investigation  of  the  influence  of  interlace  artifacts  on  the  performance 
of  a  statistical  classifier.  The  use  of  a  single  algorithm  to  explore  sensitivities  of  a  class  of  future 
algorithms  to  interlace  ran  into  technical  difficulties  to  be  explained  in  detail  later  in  this  report. 
The  results  were  thought  to  be  too  algorithm  specific  as  aUuded  to  above.  After  the  examination 
of  interlace  issues  using  a  statistical  classifier,  a  more  general  approach  was  chosen  to  examine 
sampling,  SNR,  detector  size/shape,  nonuniformity,  and  also  to  reexamine  interlace. 

While  the  study  was  directed  at  examination  of  FPA  parameters,  it  could  not  be  earned  out 
without  choosing  aFLIR  system  design  or  set  of  designs.  FPA  designs  must  be  examined  while 
considering  the  array  as  a  component  of  a  FLIR  system.  The  optimal  FPA  will  be  a  function  of 
many  other  FLIR  system  components. 

The  portion  of  the  study  which  dealt  with  the  influence  of  interlace  using  a  statistical  classifier  is 
dealt  with  separately  in  Section  3  since  the  approach  was  different  from  the  rest  of  the  study. 
Section  4  covers  the  influence  of  sampling,  S/N  and  detector  size/shape,  and  looks  again  at 
interlace.  Section  6  summarizes  work  on  a  companion  project  dealing  with  the  influence  of 
interlace  misregistration  on  trackers.  Conclusions  are  presented  in  Section  7. 
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Section  3  Influence  of  Interlace  on  a  Statistical  Classifier 


3.1  IMAGERY  DISTORTIONS  RELATED  TO  INTERLACE 

Figure  2  illustrates  three  different  interlace  inechaiusins  which  contribute  to  distortion  of  target 
shape  within  a  single  frame  which  is  composed  of  two  interlaced  fields.  They  are  due  to  a 
bidirectional  parallel  scan  mechanism,  a  unidirectional  parallel  scan  and  a  staring  array  with  odd 
and  even  field  readout  separated  by  1/2  the  total  frame  time.  Other  interlace  configurations  such 
as  serial-parallel  are  possible  but  are  not  candidates  for  second  generation  FPAs  and,  therefore, 
are  not  considered  here. 


^  FIELD  1 
FIELD  2 

^  INTERFIELD  MOTION 


DETECTORS  BELONGING  TO 


FIELD  1 


I  I  FIELD2 


Figure  2.  Motion  of  the  image  across  a)  a  bidirectionally  interlaced  FPA, 
b)  a  unidirectionally  interlaced  array  and  c)  an  interlaced  staring  array 
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In  order  to  understand  the  reasons  for  distortion  of  a  target  due  to  interlacing  it  is  important  to 
consider  the  time  interval  between  different  fields  of  a  single  frame  of  imagery.  For  an  interlaced 
staring  sensor  the  even  fields  integrate  signal  for  1/60  of  a  second  and  then  read  out,  after  which 
the  odd  fields  integrate  and  read  signal  out.  Any  translation  or  rotation  of  the  line  of  sight  (LOS) 
of  the  FLIR  (this  includes  jitter),  misregistration  of  the  scan  mechanism,  or  motion  of  the  target 
during  the  interval  between  fields  will  shift  the  target  image  in  one  field  relative  to  the  other. 
When  the  fields  are  combined  into  a  single  frame,  a  distortion  of  the  target  will  result.  The  final 
form  of  the  distortion  will  depend  upon  the  direction  of  motion  of  one  field  with  respect  to  the 
other. 

If  there  were  any  jitter  in  the  LOS  after  correction  by  gimbals  and  fine  stabilization  mechanism, 
the  interaction  of  the  residual  jitter  and  the  interlace  could  combine  to  produce  target  distortions. 
For  example,  if  jitter  moves  the  sensor  LOS  up  toward  the  end  of  the  scan  of  field  1 ,  as  is  shown 
in  Figure  3,  the  edges  of  the  target  which  do  not  lie  parallel  to  the  direction  of  motion  will 
become  jagged  and  the  aspect  ratio  of  targets  will  change  as  will  the  square  in  the  figure.  If  the 
jitter  were  to  move  the  LOS  along  the  direction  of  scan,  as  in  Figure  4,  again  the  edges  of  targets 
would  become  distorted.  This  is  the  same  type  of  distortion  which  would  result  if  a  tactical  target 
were  travelling  rapidly  parallel  to  scan.  This  same  type  of  shearing  would  be  apparent  if  the 
velocity  of  the  scene  presented  to  an  interlaced  imager  was  large,  as  is  the  case  when  the  FLIR 
LOS  is  directed  perpendicular  to  the  platform  velocity.  This  would  be  the  case  when  a  pilot  steers 
a  head-slaved  FLIR  perpendicular  to  the  line  of  motion  in  a  low  flying  aircraft. 


field  1 
field  2 


Figure  3.  Distortions  caused  by  a  simple  vertical  motion  of  the  LOS  between  field  1  (solid)  and 
field  2  (dashed),  b)  The  resulting  frame  showing  jagged  edges  and  aspect  ratio  variation 
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Figure  4.  Distortions  caused  by  a  simple  horizontal  motion  of  the  LOS  between  field  1  (solid)  and 
field  2  (dashed),  b)  The  resulting  frame  showing  jagged  edges  and  aspect  ratio  variation 


For  bidirectionally  scanned  arrays  (Figure  2a)  the  time  interval  between  scanning  across  an 
edge  of  a  target  in  one  field  and  stepping  down  and  scanning  back  across  the  same  edge  will 
depend  upon  the  position  in  the  field  of  view.  The  distortions  within  a  frame  will  therefore  be 
dependent  upon  their  position  in  the  field  of  view. 

Naturally,  the  actual  distortions  will  be  more  complicated  than  those  mentioned  above,  being  a 
complex  function  of  the  scan  velocity  and  frequency  spectrum  of  the  residual  jitter.  Residual  high 
frequency  jitter  will  act  to  displace  the  system  LOS  within  a  field.  For  a  staring  system  which 
integrates  for  a  ftame  time,  this  will  result  in  a  blurring.  For  a  parallel  scanning  system  the 
different  displacements  of  the  LOS  along  a  scan  will  produce  waves,  stretches,  and  warping 
along  the  scan  line.  The  two  types  of  jitter  described  in  Figure  3  and  Figure  4  will  serve  as 
paradigms  of  simple  distortion  with  which  we  will  probe  the  magnitude  of  the  interlace  effects  on 
algorithm  performance. 

Distortions  of  a  target  when  measured  in  pixels  will  be  a  greater  percentage  of  total  target 
subtense  at  greater  ranges.  Thus,  even  distortions  which  are  sometimes  thought  to  be  small  (those 
on  the  order  of  just  a  few  pixels),  become  increasingly  important  at  longer  ranges. 
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3J2  HORIZONTAL  MISREGISTRATION  OF  INTERLACED  FIELDS 


The  imagery  used  in  this  experiment  consisted  of  terrain  board  imagery  of  21  aspects  of  two 
vehicles,  an  M35  tmck  and  an  Ml  tank,  at  simulated  ranges  of  3500  and  4500  meters.^ 
Horizontal  misregistration  of  interlaced  fields  was  simulated  by  shifting  all  odd  lines  in  each 
image,  corresponding  to  the  distortions  illustrated  in  Figure  4.  Five  sets  of  shifted  images 
corresponding  to  pixel  shifts  of  0.5, 1.0, 1.5, 2.0, 2.5  were  created  in  this  manner.  Together  with 
the  unshifted  imagery  this  resulted  in  a  total  of  504  images. 

The  Hughes  ADASIM  algorithm  was  then  used  to  segment  the  targets  from  the  imagery  and 
extract  a  set  of  20  features  from  each  segmentation.^-^  Examples  of  the  unshifted,  shifted  and 
segmented  imagery  are  shown  in  Figure  5.  The  features  were  then  compared  and  analyzed  in 
different  ways;  only  the  most  productive  procedures  will  be  described  here. 


Figure  5.  Examples  of  unshifted,  shifted,  and  segmented  images  used  in  the  experiment. 


The  percent  change  in  feature  value  between  nonshifted  and  shifted  images  averaged  over  all  21 
aspects  was  examined.  Target  features  changed  as  a  result  of  the  pixel  shifting,  but  in  no  specific 
pattern.  There  was  no  gradual  increase  or  decrease  in  the  target  feature  values  from  non-shifted  to 
shifted  images. 

The  spread  in  feature  value  due  to  vehicle  aspect  was  also  examined  versus  pixel  shift  as  show  in 
Figure  6.  In  most  cases  the  spread  in  feature  value  due  to  vehicle  aspect  was  greater  than  the 
variation  in  feature  value  due  to  pixel  shift.  Although  this  result  appears  to  indicate  that  the 
influence  of  pixel  shifting  was  small,  the  comparison  could  be  misleading  if  extrapolated  to  final 
performance  of  the  algorithm.  Target  classification  decisions  are  made  using  aU  20  features,  not 
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just  a  single  feature,  and  because  of  this  the  appropriate  comparison  should  be  made  in  20 
dimensional  feature  space. 


Figure  6.  Variation  of  feature  value  (iargest  and  smaliest  values  and  mean) 
for  the  truck  and  tank  versus  pixel  shift 


Due  to  the  difficulty  of  illustrating  results  in  a  20  dimensional  space,  the  Bhattacharyya  distance^ 
was  measured  between  feature  values  which  had  been  averaged  over  aspect.  The  Bhattacharyya 
distance  is  a  measure  on  Euclidean  n-dimensional  space.  For  uncorrelated  features  the 
Bhattacharyya  distance  can  be  thought  of  as  a  result  of  a  change  of  coordinates:  the  origin  of  n- 
space  is  translated  to  the  vector  whose  i*  component  is  the  mean  of  the  i*  feature,  and  the  i* 
coordinate  axis  has  its  scale  given  by  the  standard  deviation  of  the  distribution  of  the  ith  feature. 
Those  features  between  which  there  was  the  greatest  distance  were  assumed  to  be  the  best  target 
discriminators.  The  top  three  features  for  each  range  were  determined  in  this  manner,  and  feature 
values  for  each  aspect  of  the  tank  and  truck  were  plotted  in  the  three  dimensional  space  having 
those  features  which  were  identified  to  be  the  best  discriminators  as  dimensions  in  order  to  look 
for  trends. 

Figure  7  and  Figure  8  show  examples  of  the  results  for  integer  pixel  shifts.  One  might  expect  the 
interlace  to  move  the  distribution  systematically.  Overlaying  the  shifted  and  nonshifted  results 
does  not  appear  to  show  any  systematic  motion  or  distortion  of  the  distribution.  Thus  from  this 
experiment  we  are  unable  to  quantify  the  performance  degradation  due  to  different  amoimts  of 
interlace  misregistration. 
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Figure  7.  Three  dimensional  feature  space  representation  of  21  different  aspects  of  the  tank  and 
truck  at  3500  meters  for  no  pixel  shift  (top),  one  pixel  shift  (middle)  and  two  pixel  shift  (bottom) 
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Since  the  approach  using  feature  values  ran  into  difficulties,  we  changed  our  approach  and  turned 
our  attention  to  the  final  decision  of  the  algorithm.  The  algoriflim  used  a  binary  decision  tree  for 
final  classification  of  the  target.  The  shifting  of  fields  due  to  interlace  could  possibly  alter  the 
final  decision  of  the  classifier.  Variation  of  a  feature  value  due  to  interlace  above  or  below  a 
specified  threshold  would  alter  the  path  followed  along  the  decision  tree.  There  are  124  terminal 
nodes  on  the  decision  tree,  24  of  which  classify  the  target  as  a  tank.  Since  many  of  the  terminal 
nodes  of  the  decision  tree  can  result  in  the  same  decision,  an  alteration  in  path  down  the  tree 
caused  by  field  shifting  can  result  in  either  1)  the  target  classification  being  the  same  as  the  non- 
shifted  classification  (path  b  in  Figure  9)  or  2)  die  target  classification  being  different  (path  c  in 
Figure  9). 


a)  unshifted 

b) ,  c)  possible  shifted  results 


Figure  9.  Possible  paths  taken  down  a  decision  tree.  The  solid  line,  labelled  path  a,  could 
represent  the  path  resulting  from  the  noninterlaced  image.  Interlace  could  result  in  a  path 
labelled  path  b  which  leads  to  the  same  classification  at  a  different  terminal  node  or 
a  path  labelled  c  which  leads  to  a  different  classification. 


The  final  classification  of  the  shifted  target  can  be  the  same  as  the  nonshifted  target  in  two  ways: 
1)  by  resulting  in  the  same  terminal  node  or  2)  by  resulting  in  a  different  terminal  node  with  the 
same  classification.  We  examined  the  percent  of  correct  classifications  at  any  terminal  node 
(Table  1)  and  the  percent  of  correct  classifications  at  the  same  terminal  node  (Table  2). 

Thble  1.  Percent  of  correct  classifications  at  any  terminal  node 


target 

range 

no  shift 

1.0  pixel  shift 

2.0  pixel  shift 

tank 

3500m 

90% 

85% 

86% 

tank 

4500m 

90% 

81% 

76% 
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Table  2.  Percent  of  correct  classiflcations  at  the  same  terminal  node  as  the  unshifted  image 


target 

range 

1 .0  pixel  shift 

2.0  pixel  shift 

tank 

3500m 

81% 

76% 

truck 

3500m 

76% 

76% 

tank 

4500m 

48% 

52% 

truck 

4500m 

57% 

57% 

One  notices  from  Table  1  a  pronounced  influence  of  range.  As  one  would  expect,  the  distortions 
due  to  interlace  become  large  relative  to  the  size  of  the  target  at  greater  ranges,  resulting  in  an 
increase  in  misclassifications.  What  we  are  not  able  to  explain  is  the  improvement  in 
classification  for  the  two  pixel  shift  over  the  one  pixel  shift  at  3(XX)  meters.  Table  2  indicates  that 
there  were  changes  in  the  path  down  the  binary  decision  tree  due  to  the  simulated  interlace 
misregistration.  Also  evident  is  the  influence  of  range  as  in  Table  1. 

3.3  VERTICAL  MISREGISTRATION  OF  INTERLACED  FIELDS 

A  different  approach  was  tried  for  the  investigation  of  the  influence  of  vertical  scan 
misregistration  due  to  interlace.  The  same  set  of  images,  21  aspects  of  the  two  targets  Ml  and 
M35  at  3500  and  4500  meters,  was  used  as  a  starting  point  for  the  testing.  From  each  of  the  84 
images  a  set  of  10  images  was  created  by  vertically  shifting  the  odd  field  a  random  amount  from 
0.0  to  2.0  pixels.  For  noninteger  shifts  a  linear  interpolation  between  adjacent  rows  in  the  original 
image  was  used  to  determine  the  pixel  value  for  the  odd  field. 

The  strategy  for  this  segment  of  the  testing  was  to  measure  the  Bhattacharyya  distance  between 
the  two  classes  of  targets  and  use  it  to  give  an  estimate  of  the  upper  bound  on  the  probability  of 
classification.  Hughes  ADASIM  was  used  to  generate  features  for  calculating  the  Bhattacharyya 
distance.  This  statistical  measure  uses  the  mean  feature  vector  and  the  covariance  matrix  of  the 
features  for  each  of  the  two  target  classes.  The  Bhattacharyya  distance  assumes  a  Gaussian 
distribution  of  features  in  feature  space.  The  Gaussian  distribution  is  also  an  assumption  which  is 
common  to  most  statistical  classifiers.  Although  this  is  not  an  accurate  assumption,  it  is  important 
that  the  assumptions  made  in  the  test  method  be  the  same  as  those  in  the  family  of  algorithms 
under  test  in  order  that  the  test  results  reflect  the  performance  of  the  general  class  of  algorithms. 

The  class  separability  for  a  noninterlaced  system  was  estimated  by  using  the  Bhattacharyya 
distance  between  the  21  aspects  of  the  Ml  and  the  21  aspects  of  the  M35.  The  separability  for  an 
interlaced  system  was  estimated  by  calculating  the  Bhattacharyya  distance  between  the  distorted 
silhouettes  of  the  21  aspects  of  the  Ml  and  the  21  aspects  of  the  M35.  Since  it  was  not  known 
how  many  samples  of  the  distorted  images  were  needed,  the  experiment  was  repeated  using  a 
larger  number  of  samples  each  time. 

The  results  of  the  experiment  are  given  in  Figure  10.  The  horizontal  axis  in  the  graph  indicates 
the  number  of  images  used.  The  vertical  axis  shows  the  loglO  of  the  probability  of  error 
estimated  by  the  Bhattacharyya  distance.  The  probability  of  error  approaches  the  asymptotes 
drawn  as  dotted  lines  in  Figure  10.  The  degradation  in  performance  due  to  an  interlaced  system  is 
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represented  by  the  difference  in  the  probability  of  enor  between  the  noninterlaced  case  and  the 
asymptotes  for  each  range. 


Nurrtw  of  imagoc  with  random  varticaJ  shift 
intariaOKj  234  567  89  10 


Figure  10.  Results  of  the  vertical  misregistration  test  showing  the  log  of  the  probability  of 
error  versus  the  number  of  images  used  to  calculate  the  Bhattacharyya  distance 


The  point  at  which  the  response  curves  approach  the  asymptotes  represents  the  number  of 
interlaced  images  (of  each  target  at  a  specific  aspect)  required  of  future  experiments.  From  Figure 
10  it  would  seem  that  six  to  ten  images  with  interlace  distortion  would  produce  about  the  same 
separability  results  as  many  thousands  of  images  with  the  same  distortion. 

The  first  thing  that  one  notices  about  the  results  is  that  the  final  probability  of  error  is  estimated 
to  be  very  small.  It  is  thought  that  this  is  the  case  because  of  the  very  small  target  set  In  order  to 
get  more  realistic  probabilities  of  error,  it  would  be  necessary  to  run  the  experiment  on  a  larger 
data  set  size. 

Another  unexpected  result  of  this  testing  is  that  the  separability  increases  for  the  longer  range 
targets.  Intuitively  we  know  that  at  longer  ranges  the  probability  of  error  should  increase  rather 
than  decrease.  Although  this  may  be  true  in  general,  it  is  possible  in  some  limited  cases  for  the 
converse  to  be  true.  To  demonstrate  this  phenomenon,  this  portion  of  the  experiment  could  be 
repeated  for  a  larger  target  set  with  more  ranges.  Instead  of  continuing  with  this  approach  to  the 
problem,  however,  work  on  a  more  general  approach  which  related  more  directly  to  the 
performance  of  future  algorithms  was  begun. 
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Section  4  General  Correlation  Approach 


Some  of  the  problems  relating  to  the  use  of  a  specific,  “first  generation”  taiget  recograuon 
algorithm  for  investigation  of  the  perfonnance  of  future  algorithms  have  already  been  direussed. 
Other  technical  difficulties  were  encountered  in  carrying  out  the  test.  For  example,  in  trying  to 
determine  the  influence  of  sampling  rate  on  algorithm  perfonnance,  we  had  problems  using 
imagery  small  enough  for  the  algorithm  to  accept.  For  high  sampling  rates  at  medium  ranges,  the 
size  of  the  target  expressed  in  pixels  was  too  large  to  fit  in  the  ADASIM  window,  within  which 
the  target  is  segmented  and  features  are  measured. 

There  were  other  obstacles  to  using  ADASIM.  In  order  to  get  results  which  were  truly 
representative  of  the  optimal  perfonnance  for  each  sampling  rate,  the  algorithm  would  have  to  be 
trained  on  a  laige  data  set  of  imagery  with  that  sampling  rate.  Every  change  to  the  FLIR  system 
design  would  require  that  the  algorithm  be  retrained,  something  which  would  require  an 
enormous  amount  of  imagery  and  time.  It  was  factors  such  as  these  which  led  to  the  change  in 
approach  to  the  problem. 

The  new  approach  chosen,  depicted  in  Figure  11,  was  composed  of  two  basic  operations,  1)  the 
simulation  of  imagery  from  FLIRs  with  different  designs  and  2)  an  interframe  measurement 
comparing  the  final  simulated  imagery  with  the  starting  imagery.  The  interframe  measure  chosen 
was  linear  correlation  which  is  similar  to  the  matching  operation  performed  by  some  template 
matching  algorithms  in  their  classification  phase.  This  approach  assumes  that  an  area  of  interest 
operation  has  already  been  performed  and  has  indicated  the  presence  of  a  possible  target 


Figure  11.  General  correlation  approach 


Imagery  of  a  target  was  first  passed  through  a  simulated  atmosphere  after  which  it  was  piped 
through  the  FLIR  image  chain.  Elements  of  the  FLER  image  chain,  the  optics,  detector,  noise, 
nonuniformity,  and  postfilter  along  with  restoration  were  the  variables  in  this  experiment  A 
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linear  correlation  was  performed  between  the  starting,  high  resolution  image  and  the  filtered 
image  which  had  been  reconstructed  to  the  same  size  as  the  starting  image  using  piecewise  cubic 
reconstruction.  A  correlation  surface  was  formed  and  the  peak  of  the  surface  was  used  as  a 
measure  of  the  quality  of  the  degraded  image.  It  was  also  taken  to  be  indicative  of  the  quality  of 
the  imager.  Since  the  quality  was  measured  using  a  procedure  which  is  similar  to  template 
matching,  it  is  believed  that  comparison  of  the  correlation  peak  values  due  to  several  simulated 
imagers  is  representative  of  the  relative  performance  of  future  ATR  algorithms,  especially  those 
which  rely  on  template  matching. 

In  fact,  an  argument  can  be  made  that  the  measured  correlation  values  reflect  an  upper  bound  on 
performance  of  this  type  of  algorithm.  The  template  or  model  which  is  being  matched,  in  this 
case  the  high  resolution  starting  image,  is  as  good  a  model  of  the  target  as  one  can  possibly  get. 
The  target  aspect  is  exactly  that  of  the  template.  The  choice  of  target-to-template  orientation  is 
also  perfect,  as  is  the  choice  of  vehicle  state  (hot  versus  cold  engine,  hot  versus  cold  treads,  etc.). 
Any  other  choice  of  the  target  template  (reference  image)  would  result  in  a  lower  correlation 
value.  Because  correlation  is  thought  to  be  related  to  relative  performance  of  ftiture  algorithms,  it 
is  used  as  the  basis  of  comparison  for  future  FLIR  designs. 

The  correlation  measure  has  some  disadvantages.  It  is  a  global  measure,  and,  as  such,  it  may  not 
directly  reflect  the  performance  of  algorithms  which  rely  upon  specific  features  of  a  target 
appearing  in  an  image,  for  instance,  an  algorithm  which  looks  for  a  gun  barrel  before  it  declares  a 
target  to  be  a  tank.  For  algorithms  such  as  these,  a  local  measure  of  image  quality  might  be  more 
appropriate. 

Correlation  is  also  difficult  to  relate  directly  to  range  or  some  other  typical  figure  of  merit  which 
can  be  used  to  quantify  performance.  The  failure  of  our  efforts  to  use  a  type  of  cross  correlation 
to  relate  results  to  range  will  be  discussed.  The  calculation  of  the  correlation  surface  is  computer 
intensive,  and  since  a  study  like  this  should  be  done  on  a  large  ensemble  of  images  in  order  to  get 
good  statistics,  the  computer  time  required  to  perform  the  test  is  enormous. 

Despite  the  drawbacks  of  the  correlation  measure,  it  was  thought  to  be  the  most  representative  of 
future  algorithm  performance  and  was  used  throughout  the  remainder  of  the  study.  Similar 
approaches  have  been  taken  by  contractors  in  comparable  efforts®*^  however,  this  investigation  is 
by  far  the  most  exhaustive. 

4.1  THE  IMAGING  CHAIN 
4.1.1  Targets 

Imagery  of  the  targets  used  as  a  reference  set  in  this  experiment  is  reproduced  in  Figure  12 
through  Figure  15  along  with  a  histogram.  The  imagery  was  taken  by  a  radiometer’  at  ranges  less 
than  100  meters.  The  images  were  480  x  512  pixels  x  8  bits.  We  chose  images  with  several  pixels 
across  the  tai;get  for  the  reference  image  set  so  that  we  could  consider  the  sampling  in  the 
reference  image  to  be  negligible  when  compared  with  the  much  coarser  sampling  to  be  simulated 
later  in  the  image  chain. 
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Figure  14.  BMP  front  and  the  corresponding  histogram 
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Figure  15.  BMP  right  front  and  the  corresponding  histogram 
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4.U  Atmosphere 


A  simple  Beer’s  law  atmosphere  was  used  to  simulate  the  degraded  contrast  which  would  result 
from  a  typical  atmosphere.  The  reduced  contrast  when  combined  later  in  the  FLIR  imaging  chain 
with  a  signal  independent  noise  results  in  a  decreased  signal  to  noise.  The  form  for  the 
attenuation  used  was 


Apparent  ~  I  ^Apparent  ~  ^Ambient  l~l  ^Inherent  ~  ^Ambient  I  (*<0 1) 

where  is  the  ambient  temperature  which  was  chosen  to  be  equal  to  the  mean  of  the 

image,  Ty^ppgfgnt  is  the  temperature  of  a  scenel  in  the  reference  image  after  passing  through  the 
atmosphere,  is  the  scenel  temperature  before  passing  through  the  atmosphere,  R  is  the 

range  in  km,  and  sigma  is  the  attenuation  coefficient  in  km'^  (The  term  scenel  will  be  used 
throughout  this  report  to  denote  a  picture  element  in  the  refermce  image.  A  separate  teim  is  used 
to  distinguish  it  from  a  picture  element  in  an  image  after  application  of  the  FLIR  simulation 
which  will  be  referred  to  as  a  pixel.)  Two  values  of  sigma  were  used,  the  first  being  0.3  which 
corresponds  to  a  relatively  good  atmosphere  with  approximately  74%  transmission  per  km  and 
the  second  value  equal  to  0.6  or  approximately  54%  per  km.  The  ambient  temperamre  in  all  cases 
was  chosen  to  be  the  mean  temperature  of  the  image.  The  action  of  the  atmosphere  was  to 
contract  the  histogram  of  the  scene  about  the  mean  value  of  the  taiget  Examples  of  imagery  at  3 
and  6  km  with  no  atmosphere,  the  “good”  and  the  “bad”  atmosphere,  along  with  their  respective 
histograms  are  shown  in  Figure  16  through  Figure  19. 
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Figure  16.  Example  of  imagery  at  3km  with  "good”  atmosphere 
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Figure  19.  Example  of  imagery  at  6km  with  “bad”  atmosphere 


4.13  FUR 


The  components  of  the  FLIR  image  processing  chain  and  the  simulation  of  each  are  covered  in 
the  following  sections. 


4.1.4  Optics 


The  optics  of  the  simulated  FLIR  were  assumed  to  be  difiraction  limited.  Image  formation  on  the 
FPA  was  simulated  by  convolving  the  atmospherically  attenuated  image  with  the  diffraction 
pattern  from  a  circular  aperture  which  has  the  form:*»® 


hopt(^)  =  2 


I  Jl(ii(rlro)) 

1  n(rlrQ)  I 


(EQ2) 


where  Jl(r)  is  the  Bessel  function  of  order  1, 

r  =  V  +  y^  (EQ  3) 


and 

ro=  1.22AR/Z) 


(EQ4) 


where  X  and  y  are  coordinates, 

X  =  wavelength  of  the  IR  radiation  (10  pm),  and 
D=  diameter  of  optics  (0.203m). 
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By  substituting  the  focal  length  for  R  above,  the  radius  of  the  Airy  disk  at  the  focal  plane  was 
found  to  be  27.5  pm.  To  project  the  radius  of  the  Airy  disk  into  object  space,  (with  respect  to  the 
reference  image)  one  substitutes  the  simulated  range  to  target  for  R  (3, 4.5,  or  6  km),  resulting  in 
18, 27  and  36  cm  respectively.  In  order  to  make  the  simulation  more  tractable,  hopt(r)  was 
truncated  at  r=:ro. 

4.1.5  Detector,  temporal  integration  and  sampling 

The  image  gathering  operation  of  a  detector  which  scans  across  an  image  plane  can  be  separated 
mathematically  into  a  term  which  characterizes  the  spatial  response  of  the  detector,  a  term  which 
describes  the  temporal  integration  of  the  detector  while  scarating,  and  a  term  which  defines  the 
sample  spacing.  The  detector  spatial  response  here  is  assumed  to  be  a  rect  function.  The  fimction 
which  indicates  the  temporal  extent  of  a  sample  is  also  a  rect  function.  Sample  spacing  is 
described  using  a  brash  function,  a  regular  2-D  array  of  Dirac  delta  functions. 

The  two  rect  functions  are  first  convolved  to  produce  a  trapezoid  or  triangle  function.  This 
trapezoid,  after  being  appropriately  sized  with  respect  to  the  reference  image,  could  then  be 
convolved  with  the  whole  image  in  order  to  account  for  the  blurring  due  to  the  detector.  The 
brash  function  could  then  operate  on  the  image  so  that  samples  are  chosen  at  the  appropriate 
intervals.  Instead  of  actually  performing  all  operations  separately,  it  requires  far  less  computation 
to  combine  the  convolution  and  sampling  into  one  operation  by  simply  placing  the  trapezoid 
functions  at  the  appropriate  sample  spacings  and  summing  the  weighted  contributions  from  the 
trapezoid. 

There  are  three  separate  grids  which  come  into  play  in  this  sampling.  First,  there  is  the  grid 
defining  the  reference  image  scenels,  which  have  a  particular  angular  separation  at  a  given  range. 
The  second  grid  is  one  which  describes  the  detector  spatial  response  combined  with  integration 
while  scanning.  Each  element  of  the  detector  array  is  projected  out  to  the  simulated  range  and 
overlaid  onto  the  grid  of  the  reference  image.  Bilinear  interpolation  is  performed  to  compute  the 
input  image’s  response  to  each  detector  template  element.  These  values  are  summed  over  the 
entire  detector  to  form  one  sample.  The  third  grid  corresponds  to  the  sampled  image. 

Normally  the  sampling  operation  is  begun  by  placing  the  detector  template  exactly  in  the  upper 
left  hand  comer  of  the  reference  image.  The  influence  of  sample-scene  phasing  can  be 
investigated  by  simply  starting  the  sampling  operation  shifted  by  a  firaction  of  a  detector  template 
with  respect  to  the  reference  image.  This  was  done  at  rune  evenly  spaced  positions  within  the 
unit  ceU  formed  by  four  adjacent  samples  (See  Rgure  20). 


22  The  Influence  of  Focal  Plane  Array  Design  Parameters  on  Future  Automatic  Target  Recognizer  Performance 


H  X  X  • 

XXX 

XXX 


Figure  20.  Unit  cell  of  the  sampling  grid  (dots)  and  different  positions  (Xs) 
chosen  to  simulate  the  effects  of  sample-scene  phasing. 


4.1.6  Interlace 

The  influence  of  interlace  misregistration  was  previously  investigated  (see  Section  3  on  page  5) 
using  terrain  board  imagery  which  had  every  other  line  of  imagery  shifted  either  vertically  or 
horizontally.  The  shifted  images  were  then  operated  on  by  the  ADASIM  algorithm.  The  approach 
could  be  questioned  since  inteipolation  was  required  to  form  one  field  and  not  the  other.  This 
would  filter  the  interpolated  field  and  not  the  other  field,  perhaps  influencing  the  results. 

A  different  approach  using  the  FUR  simulation  was  used  in  this  portion  of  the  experiment.  In  a 
marmer  similar  to  that  used  to  produce  imagery  with  different  sample  scene  phases,  imagery  with 
specific  amounts  of  interlace  misregistration,  both  vertically  and  horizontally,  were  produced. 

The  operation  did  not  filter  one  part  of  the  image  more  than  another  and  thus  was  thought  to  be 
more  realistic. 

4.1.7  Noise 

In  real  imaging  systems  some  deteaors  are  noisier  than  others.  For  parallel  scanned  imaprs  this 
produces  a  row  to  row  variation  in  the  amount  of  noise  in  the  final  image.  For  this  expenment  we 
have  specified  noise  distributions  for  eadi  row  of  the  image.  For  the  first  thrw  noise  cases  the 
noise  along  a  particular  row  was  chosen  randomly  from  a  Gaussian  distribution  with  a  mean  of 
zero.  The  standard  deviation  of  the  distribution  for  each  row,  Op  was  also  randomly  chosen  from 
a  Gaussian  distribution  with  nonzero  mean  p  and  standard  deviation  c.  Parameters  summarizing 
the  three  noise  levels  investigated  are  presented  in  Table  3.  For  the  fourth  noise  case  Oj.  was 
diosen  to  be  a  constant  for  the  whole  image,  the  value  of  the  constant  varied  with  detector  area. 
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Table  3.  Parameters  of  the  Gaussian  distribution  of  Oj. 
corresponding  to  noise  added  expressed  in  quantization  levels 

Noise  Level  m  s 

1  1  2 

2  2  2 

3  4  4 

The  mean  values  of  the  distributions  were  chosen  to  span  typical  FLIR  design  philosophies.  That 
is,  some  FLIR  designers  indicate  that  they  design  a  FLIR  so  that  the  magmtude  of  the  least 
significant  quantization  level  is  equal  to  the  system  Noise  Equivalent  Temperature  difference 
(NET).^®  Other  designers  advocate  quantizing  the  dynamic  range  so  that  there  are  four 
quantization  levels  within  the  temperature  span  of  one  NET.  Their  arguments  for  one  choice  or 
another  center  around  the  required  thermal  dynamic  range  as  well  as  the  visibility  of  noise  after 
the  human  visual  system  has  spatially  and  temporally  integrated  the  scene. 

Imagery  which  is  representative  of  the  three  noise  levels  is  shown  in  Figure  21. 


Figure  21.  Representative  imagery  with  noise  a)  level  1,  b)  level  2,  c)  level  3 
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Another  experiment  was  perforated  where  it  was  assumed  that  the  system  was  a  BLIP 
(Background  Limited  F^otodetector)  system. 

For  tactical  infrared  imagers  the  ultimate  sensitivity  of  the  imaging  system  is  achieved  when  the 
dominant  noise  source  results  from  the  fluctuation  in  the  number  of  photons  arriving  from  the 
scene.ii  For  a  system  which  is  limited  by  this  type  of  noise,  called  a  BLIP  limited  (Background 
T  imitp/t  Photodetector)  system,  a  larger  detector  is  advantageous  from  a  SNR  standpoint.  The 
probability  of  arrival  of  a  photon  at  a  detector  can  be  approximately  described  by  a  Poisson 
distribution.  The  SNR  measured  after  the  detector  is  the  square  root  of  the  mean  number  of 
photons  arriving  during  a  particular  time  interval.  A  larger  mean  number  of  photons  will  he 
incident  upon  a  detector  with  larger  angular  subtense  (or  larger  physical  area  if  the  optics  is  kept 
constant),  resulting  in  a  higher  SNR  in  the  final  image. 

For  a  BLIP  system  the  signal  is  taken  to  be  equal  to  the  mean  munber  of  photons,  which  in  turn  is 
proportional  to  the  detector  area.  The  variance  of  the  Poisson  process  is  the  mean,  and  the  RMS 
noise  level  is  the  square  root  of  the  mean  number  of  photons.  The  signal  to  RMS  noise  level  is 
then  proportional  to  the  square  root  of  the  mean,  and  thus  proportional  to  the  square  root  of  the 
detector  area.  In  this  experiment  the  variation  of  SNR  with  changing  detector  area  was 
accomplished  by  varying  the  amount  of  noise  added  to  the  image. 

This  improvement  in  SNR  for  larger  detectors  will  act  to  improve  the  final  correlation  between 
the  simulated  and  reference  images.  At  the  same  time  the  decreased  resolution  due  to  the  larger 
detectors  will  degrade  the  correlation  indicating  that  optimal  system  design  ultimately  involves  a 
trade-ofi"  between  system  sensitivity  and  resolution.  This  improvement  in  system  SNR  due  to 
choice  of  a  larger  detector  could  be  used  to  increase  the  system  dynamic  range,  or  could  be  used 
to  decrease  the  size  of  a  quantization  level  (as  expressed  in  degrees  K).  Both  modifications  to  the 
system  design  would  require  increased  system  bandwidth  however. 

For  the  cases  where  noise  was  treated  independently  of  detector  area  and  for  the  BLIP  detectors 
where  the  noise  was  tied  to  the  detector  area  the  mean  values  of  the  distributions  describing  the 
variances  were  chosen  to  span  typic^  FLIR  design  philosophies.  That  is,  some  contractors  have 
indicated  that  they  have  designed  a  FLIR  so  that  the  magnitude  of  the  least  sigmficant 
quantization  level  is  equal  to  the  system  NET.  Others  advocate  quantizing  so  that  there  are  four 
quantization  levels  within  the  temperature  span  of  one  NET.  Their  arguments  for  one  choice  or 
another  center  around  the  visibUity  of  noise  after  the  human  has  spatially  and  temporally 
integrated  the  scene. 

4.1.8  Nonuniformity 

The  nonuniformity  being  simulated  in  this  portion  of  the  simulation  is  residual  nonuniformity  in 
the  image  after  correction.  In  general,  the  form  of  the  residual  spatial  noise  in  an  image  after 
implementation  of  a  linear  n-point  correction  scheme  is  a  complicated  fimetion  of  the  number  of 
correction  points  and  the  temperature  spread  between  the  points.*^  Simulation  of  the  exact  form 
of  the  spatial  noise  vs.  input  photon  flux  for  our  application  would  have  been  an  unnecessary 
complication.  Instead  we  chose  a  much  simpler  approximate  way  to  simulate  the  residual 
nonuniformity. 
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The  nonuniformity  of  each  simulated  detector’s  response  to  signal  is  assumed  to  be  of  unity  gain 
in  all  cases.  The  offset  for  each  simulated  line  is  assumed  to  be  constant  for  the  whole  line.  The 
value  of  the  offset  is  the  result  of  a  random  draw  from  a  Gaussian  distribution  of  possible  offsets 
described  by  a  mean  of  zero  and  a  standard  deviation  of  1, 4  or  8  quantization  levels, 
corresponding  to  nonuniformity  levels  1, 2  and  3  respectively.  Table  4  gives  the  statistics  for  the 
offset  distribution  for  the  3  nonuniformity  levels. 

Table  4.  Nonuniformity  levels  simulated  and  the  corresponding  mean  and  standard  deviation 
values  quoted  in  units  of  quantization  levels  or  bits  for  gain  and  offset  distributions 


mean  of 

std.  dev.  of 

mean  of 

std.  dev. 

Level 

gain 

gain 

offset 

offset 

1 

0 

0 

0 

1 

2 

0 

0 

0 

4 

3 

0 

0 

0 

8 

The  variance  in  signal  for  a  flat  field  in  the  cross  scan  direction  is  seen  to  be  the  same  order  of 
raagniuide  as  the  variance  of  the  noise.  They  are  both  chosen  to  be  of  the  same  order  of 
magnitude  as  a  system  NET  to  which  systems  are  designed.  Of  course  actual  system  performance 
of  second  generation  prototypes  has  not  always  achieved  the  level  of  performance  specified  for 
the  system.  Examples  of  imagery  with  nonuniformity  levels  1, 2,  and  3  are  shown  in  Figure  22. 


Figure  22.  Examples  of  imagery  with  simulated  nonuniformity  a)  level  1,  b)  level  2,  c)  level  3 
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4.1.9  Reconstruction 


Once  the  image  has  been  processed  by  the  FLIR  simulation,  the  resultant  image  is  much  smaller 
than  the  reference  image  due  to  sampling.  In  order  to  compare  the  result  with  the  original,  the 
output  image  must  be  rescaled  to  the  original  image  size.  For  an  example  see  Figure  23.  The 
sampled  images  were  reconstructed  to  their  original  pixel  size  using  a  two  parameter  piecewise- 
cubic  convolution  algorithm.  The  scale  factor  used  was  the  ratio  of  input  image  size  to  output 
image  size. 


Figure  23.  Examples  of  sampled  and  then  reconstructed  imagery  (not  to  scale) 


One-dimensional  piecewise-cubic  leconstraction  is  implemented  by  convolving  the  samples 
of  a  sampled  image  g  with  a  piecewise-cubic  kernel  r  to  define  the  continuous  result  G 


oo 

G(x)=  2  g[n]r(x-n)  ,^<x<«> 

The  two  parameter  cubic  function  r  is  given  by 

(2  +  a- 6p)  |x|3  -H  (- 3-a  +  9p)  +  (1  -2p) ,  W  <  1 

^  (oc + 2P)  -  (5a + 9p)  W2  +  (8a  +  12p)  W  -  (4a + 4p),  1  <  W  <  2 

0,  otherwise 


The  Influence  of  Focal  Plane  Array  Design  Parameters  on  Future  Automatic  Target  Recognizer  Performance  27 


This  reconstruction  method  was  shown  by  Park  and  Reichenbach^^  to  minimize  mean  square 
error  at  low  frequencies  when  (x=-0.5,  p=0.0.  The  algorithm  uses  a  small  kernel  size  to  efficiently 
fit  sampled  data  to  a  piecewise  cubic  function. 

For  a  =  -  0.5,  and  P  =  0.0,  i(x)  interpolates  the  sampled  image  and  minimizes  mean  square 
error  between  reconstructed  image  and  the  original  image.  When  a  =  -0.5,  and  P  =  1/6,  r(x) 
approximates  the  cubic  B-spline.  Since  the  image  data  used  in  the  experiments  contained  sharp 
edges,  and  thus  high  frequency  information,  reconstruction  with  a  =  -0.5,  P  =  0.0  led  to  some 
reconstructed  pixels  having  negative  values,  so  the  parameter  settings  of  a  =  -0.5,  and  p  =  1/6 
were  used  in  the  experiments. 

Since  r(x)  is  non-zero  only  for  W  <  2,  only  four  points  are  needed  from  the  sampled  image  to 
reconstruct  each  intermediate  pixel.  If  the  scale  factor  for  reconstruction  is  S,  then  to  fill  in  the  jth 
point  between  adjacent  input  pixels  X  and  X+1  from  sampled  image  g: 

2 

h{SX^j)=  X  5^  +  ik;r/--k;,0^;^('LSX+lJ-LSXj-l) 
ik  =  -l  S 

The  reason  j  ranges  as  above  is  that  the  scale  factor  S  is  not  necessarily  an  integer. 

The  reconstruction  was  done  in  two  parts.  First,  the  image  was  rescaled  horizontally;  then  that 
array  was  scaled  vertically  to  achieve  the  original  image  size. 

4.2  INTERFRAME  COMPARISON  -  CORRELATION 

After  FPA  parameter  changes  were  made  to  a  FLIR  image  and  the  image  was  rescaled  to  its 
approximate  original  size,  a  method  was  needed  to  compare  the  rescaled  image  to  its  reference 
image.  Linear  correlation  provided  a  consistent  measure  for  comparison  of  the  rescaled  and 
reference  images.  The  main  reason  correlation  was  chosen  was  the  similarity  of  correlation  with 
the  operation  of  template  matching.  It  also  gave  numerical  values  which  could  be  used  to 
determine  which  FPA  parameter  changes  affected  FLIR  performance.  Correlation  values  could 
be  used  to  rank  the  relative  effects  of  the  various  FPA  parameter  changes:  samples/IFOV,  sample 
scene  phasing,  atmosphere,  detector  size  and  shape,  noise,  and  nonuniformity. 

The  linear  correlation  coefficient  r  is  given  by  the  formula: 

X  (Xi-Diyi-y) 

(Xi-xp  -^X  (yi-yf 

where  "x  is  the  sample  mean  of  the  x  ‘s  (pixel  values  of  the  reference  image),  and  y  is  the  sample 
mean  of  the  Y's  (pixel  values  of  file  rescaled  image).  The  value  of  r  lies  between -1  and  1, 
inclusive.  It  takes  on  a  value  of  1,  termed  “complete  positive  corhen  two  images  are  exact 
replicas  of  each  other.  A  value  of  r  near  0  indicates  that  the  two  images  are  uncorrelated.  When  r 
has  the  value  -1,  the  two  images  have  “complete  negative  correlation.” 
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The  correlation  method  is  extremely  calculation  intensive.  Because  the  rescaled  image  was 
smaller  than  the  reference  image,  it  was  necessary  to  shift  the  rescaled  image  over  the  reference 
image  and  calculate  individual  correlations  for  each  alignment  of  the  two  images.  This  procedure 
involved  matching  the  centroids  of  the  two  images,  shifting  the  rescaled  image  one  pixel’s 
distance  in  a  24x24  pixel  grid  around  the  centroid  of  the  reference  image,  and  using  the  given 
correlation  formula  to  calculate  a  correlation  value  at  each  new  location  of  the  rescaled  image. 
Although  this  process  involved  making  576  separate  correlations  between  the  two  images,  each 
of  which  contained  from  200,000  to  250,000  pixels,  it  insured  that  the  peak  correlation  value  was 
determined  by  the  best  alignment  of  the  two  images.  To  minimize  round  off  error,  all 
calculations  were  performed  with  floating  point  pixel  values  rounded  to  six  decimal  places.  This 
increased  the  correlation  calculation  time,  but  gave  a  more  precise  correlation  value,  especially  in 
view  of  the  fact  that  up  to  500,000  pixel  values  were  involved  in  calculating  one  correlation 
between  the  two  images. 

43  TEST  SOFTWARE 

Subroutines  were  created  to  simulate  each  of  the  different  operations  of  the  atmosphere  and 
FLIR.  An  input  image  was  operated  on  by  each  in  turn.  Finally,  correlation  was  calculated 
between  the  resulting  and  starting  images.  The  whole  procedure  was  controlled  through  the  use 
of  a  piping  system  and  experiment  files,  the  details  of  which  are  presented  in  Section  A-1.  The 
piping  system  worked  extremely  well  because  it  allowed  many  variations  of  FPA  parameter 
changes  to  be  made  while  the  experiment  was  in  progress. 
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Section  5  Results  Using  Correlation  Approach 


The  different  combinations  of  variables  tested  and  their  values  are  presented  in  Table  5.  Each 

describes  those  variable  values  used  in  a  particular  experiments  The  results  of  each  of  the 
eight  experiments  will  be  discussed  in  the  next  eight  sections.  In  all  cases  the  values  of  the 
variables  used  in  the  test  were  chosen  to  span  those  values  typical  of  first  generation  systems  and 
proposed  second  generation  FLIRs.  Specifically,  we  addressed  issues  pertinent  to  the  IRFPA 
program.  Addition  of  more  detectors,  higher  sampling  rates,  more  stringent  noise  and 
nonuniformity  specifications,  and  going  to  a  noninterlaced  design  all  tended  to  drive  up  the  cost 
of  proposed  second  generation  focal  plane  arrays. 

Table  5.  Test  variable  matrix 


TARGETS 

MBOFront 

MBORtFront 

BMPFront 

BMPRtFront 

RANGE 

3000m 

4500m 

6000m 

ATMOSPHERE 

sigma  0.0  none 

0.1 

0.2 

0.3  good 

0.4 

0.5 

0.6  bad 
OPTICS 

DETECTOR  SIZE 
IFOV-x  width  (mrads) 
0.05 
0.08 
0.12 

IFOV-y  height  (mrads) 
0.05 
0.08 
0.12 
0.16 

SAMPLES/IFOV-x&y 

1.0 

1.5 

2.0 


Noise 

Atmos¬ 

Atmos- 

Nonuni¬ 

Atmos- 

Sampling 

Sampling 

Sample 

phere 

phere& 

formity 

phere& 

Square 

Detector 

Scene 

Noise 

Nonuni 

Detector 

Asp.  Ratio 

Phase 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Y 

X 

y 

X 

y 

30  The  Influence  of  Focal  Plane  Array  Design  Parameters  on  Future  Automatic  Target  Recognizer  Performance 


Table  5.  Test  variable  matrix  (continued) 


Noise  Atmos¬ 

Atmos- 

Nonunl- 

Atmos-  Sampling  Sampling 

Sample 

phere 

phere& 

formlty 

phere&  Square  Detector 

Scene 

Noise 

NonunI  Detector  Asp.RatIo 

Phase 

SAMPLE  SCENE  PHASE 

none-1  frame 

X  X 

X 

X 

XXX 

X 

3x3-  9  frames 

X 

NOISE  4FR. 

level  1 

X 

X 

level  2 

X 

X 

level  3 

X 

X 

level  4 

X 

NONUNIFORMITY  6FR. 

level  1 

X 

X 

level  2 

X 

X 

level  3 

X 

X 

CROSSCORRELATION 

MSOFr  -  MSOFr 

X  X 

X 

X 

XXX 

X 

MSOFr  -  MSORtFr 

X 

X 

MSOFr  -  BMPFr 

X 

X 

MSOFr  -  BMPRtFr 

X 

X 

Noise 

Level 

o 

1 

1 

2 

2 

2 

2 

3 

4 

4 

Level  I 

mean  of 

std.  dev.  of 

mean  of  std.  dev.  of  offset 

gain 

gain 

offset  offset 

1 

0 

0 

0  1 

2 

0 

0 

0  4 

3 

0 

0 

0  8 

In  the  first  few  tests,  variables  were  tested  one  at  a  time,  not  only  to  determine  the  influence  a 
particular  variable  had  on  ATR  performance  but  also  to  determine  the  range  of  parameter  values 
which  began  to  have  an  appreciable  influence  on  correlation.  In  this  way  the  first  few  tests 
provided  information  which  allowed  us  to  decide  whether  certain  tests  should  be  performed,  and 
if  drey  should,  what  was  the  best  choice  of  starting  parameters  for  those  tests  involving 
combinations  of  variables.  For  those  combinations  of  variables  which  one  would  expect  to  be 
strongly  coupled,  for  example  sampling  rate  and  detector  IFOV,  values  of  the  two  variables  were 
varied  systematically  over  a  range  which  spanned  possible  first  and  second  generation  system 
designs. 

For  most  of  the  testing  the  image  of  the  M60  tank  front  was  used  as  the  reference  image.  The 
simulation  was  performed  at  ranges  of  3, 4.5  and  6  km,  thus,  spanmng  tactically  relevant  ranges. 
The  optics  F/#  was  225  for  all  cases.  Atmosphere  was  not  included  unless  specified. 
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Three  variables  explored  were  assumed  to  be  random:  noise,  nonuniformity  and  sample-scene 
phase.  That  is,  a  series  of  images  with  the  same  level  of  noise,  but  with  different  starting  seeds  in 
the  randomization  routine  which  generated  the  noise,  resulted  in  different  values  of  correlation. 
For  the  chosen  values  of  noise,  we  had  to  determine  the  spread  in  correlation  values  and  how  the 
spread  changed  with  range.  In  the  case  of  sample-scene  phase,  the  parameter  which  is  analogous 
to  the  starting  seed  of  the  random  number  generator  for  noise  is  the  starting  position  of  the 
sampling  grid  within  a  unit  cell.  For  a  real  system  the  phasing  effects  would  manifest  themselves 
whenever  the  system  LOS  was  changed. 

One  would  expect  that  noise  and  nonuniformity  might  have  a  strong  coupling  with  signal  level 
(as  determined  by  the  signal  level  in  the  reference  image  and  the  atmospheric  attenuation).  Since 
the  signal  level  decreases  with  range  due  to  atmospheric  attenuation,  it  is  expected  that  any 
appreciable  spread  in  correlation  values  due  to  noise  or  nonuniformity  would  increase  with  range. 
One  would  also  expect  that  sample-scene  phasing  would  be  closely  coupled  with  sampling  rate 
and  detector  size.  A  few  of  these  random  variables  were  investigated  in  the  first  phases  of  the 
testing  in  order  to  detennine  the  magnitude  of  their  influence  on  correlation.  If  their  magnitude 
were  determined  to  be  large,  we  then  investigated  the  coupling  between  them  and  other  variables. 


5.1  NOISE 

The  system  simulated  in  this  portion  of  the  test  had  a  square  IFOV  with  1.5  samples  per  detector 
subtense  in  both  directions.  The  optics  F/#  was  2.25.  The  same  basic  system  design  was  used  in 
several  of  the  following  experiments. 

Four  images  of  each  of  the  three  noise  levels  were  created  at  each  of  the  three  ranges  (see  Figure 
21).  Figure  24  is  a  plot  of  the  results  which  indicate  very  little  degradation  of  the  correlation  due 
to  noise  alone.  Also  note  that  the  spread  of  correlation  values  is  almost  negligible. 
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Figure  24.  Plot  of  correlation  vs.  range  for  3  noise  levels 
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Lev«l 


mean  of 
gain 


std.  dev.  of 
gain 


mean  of 
offset 


std.  dev.  of  offset 
offset 


1  0  0  1  2 

2  0  0  2  2 

3  0  0  4  4 

If  one  compares  this  result  with  those  for  the  same  system  with  no  noise,  which  is  plotted  as 
curve  2  in  Figure  32,  (see  Section  5.7  on  page  58)  the  degradation  of  correlation  with  range  is 
seen  to  be  almost  totally  due  to  optics,  detector,  and  sampling,  with  hardly  any  contribution  of  the 
noise  levels. 

5.2  ATMOSPHERE 

For  the  same  basic  system  described  in  section  5. 1  above  without  noise,  we  investigated  the 
influence  of  atmosphere.  Correlation  values  of  the  imagery  degraded  by  no  atmosphere  and  the 
good  and  bad  atmosphere  are  plotted  vs.  range  in  Figure  25.  One  notices  that  there  is  negligible 
degradation  of  the  correlation  due  to  the  atmosphere  alone. 


I  ' — - 1  ■  -r 


Figure  25.  Correlation  vs.  range  for  no  atmosphere,  “good”  atmosphere  and  “bad”  atmospheres 


We  next  tested  the  interaction  of  noise  and  the  variable  which  strongly  influenced  signal  level: 
atmosphere.  It  was  felt  that  trying  to  look  for  a  coupUng  of  noise  with  other  variables  related  to 
resolution  would  not  be  fruitful. 
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S.3  NOISE  WITH  ATMOSPHERE 


For  the  same  basic  system  and  ranges  as  in  section  5.1,  we  added  both  noise  and  atmospheric 
attenuation.  The  results  plotted  in  Figure  26  indicate  a  dramatic  degradation  of  correlation  for  the 
poor  atmosphere,  (n.b.  The  y  axes  will  change  from  one  presentation  of  exp)erimental  results  to 
the  next  in  the  following  figures.)  As  in  the  previous  experiment  dealing  with  noise,  four  images 
with  the  same  noise  level  were  created  at  each  range.  The  spread  of  correlation  values  becomes 
significant  at  longer  ranges  especially  for  the  poorer  atmosphere  where  the  signal  to  noise  is  low. 
The  imagery  becomes  almost  unrecognizable  for  low  correlation  values,  0  to  approximately  0.5 
or  0.6. 
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Figure  26.  Correlation  versus  range  for  two  different  atmospheres  and  three  different  noise  levels 


5.4  NONUNIFORMITY 

Again,  for  this  experiment  the  basic  FUR  system  design  was  kept  the  same  as  in  the  previous 
tests.  Three  different  levels  of  nonuniformity,  as  described  in  Section  4.1.8  on  page  25,  were 
added  to  the  imagery.  No  atmosphere  was  simulated.  Six  frames  of  each  nonuniformity  level 
were  created  at  each  range.  The  results  which  are  plotted  in  Figure  27  indicate  little  degradation 
in  correlation  value  except  for  the  highest  level  of  nonuniformity,  level  3.  The  spread  in 
correlation  is  significant  only  for  level  3  as  well. 

Present  FLIR  system  designs  are  tending  toward  increased  dynamic  range,  on  the  order  of  ten  to 
twelve  bits  spread  over  a  temperature  range  of  40  to  50  degrees.  Unconected  nonuniformity 
values  can  be  very  large  for  HgCdTe,  on  the  order  of  10%  of  the  total  dynamic  range.  After 
correction  the  goal  of  most  system  designs  is  to  get  the  nonuniformity  smaller  than  an  NET.  It 
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depends  on  the  contractor  design  philosophy  as  to  whether  they  tiy  to  set  an  NET  equal  to  a  least 
significant  bit  or  whether  they  try  to  digitize  the  NET.  The  level  of  nonunifomity  represented  by 
level  3  is  perhaps  close  to  what  one  might  expect  for  poorly  corrected  HgCdTe  imagery. 


For  a  human  looking  at  a  displayed  image  sequence,  the  streaking  resulting  from  uncorrected 
nonuniformity  is  emphasized  by  the  temporal  integration  of  the  eye.  On  the  other  hand,  the 
temporal  integration  of  the  eye  tends  to  minimize  the  impact  of  random  temporal  noise  in  an 
image  sequence. 
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Figure  27.  Correlation  vs.  range  for  different  levels  of  nonuniformity  alone 
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5S  ATMOSPHERE  AND  NONUNIFORMITY 

Keeping  the  basic  FLIR  design  consistent  with  previous  experiments,  we  looked  at  the  influence 
of  atmosphere  with  nonuniformity.  Figure  28  is  a  plot  of  the  results  which  are  similar  to  those  for 
the  noise  plus  atmosphere  case.  For  the  “good”  atmosphere  there  was  Utfle  degradation  of 
correlation  with  range;  however,  for  the  bad  atmosphere  the  degradation  with  range  is  even  mote 
pronounced  than  for  the  worst  noise  case. 
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Figure  28.  Atmosphere  plus  nonuniformity 


5.6  INTERLACE  MISREGISTRATION 

Using  the  basic  FLIR  design  as  in  the  previous  tests,  we  varied  the  misregistration  of  fields  as 
discussed  in  paragraph  2  of  Section  4.1.6  on  page  23  and  looked  at  die  correlation  peak  value 
versus  range.  Figure  29  and  Figure  30  show  results  of  misregistration  in  the  x  and  y  directions 
respectively.  Degradation  in  correlation  is  not  very  large  between  the  case  of  no  misregistration 
and  the  quarter  or  half  pixel  misregistration  cases  (between  line  1  and  lines  2  or  3  of  the  graph  in 
Figure  29  and  Figure  30)  in  either  the  x  or  y  direction.  However,  for  one  and  two  pixel 
misregistration  in  both  directions,  the  degradation  is  sigmficant. 

Second  generation  FLIR  systems  usually  limit  the  LOS  jitter  to  a  small  portion  of  an  IFOV 
(typically  less  than  20%).  If  this  is  the  case,  the  interaction  of  an  interlaced  scan  with  residual 
jitter  would  probably  not  produce  lai^e  misregistrations  on  the  order  of  2  pixels.  Improper  timing 
could  produce  such  an  image  defect. 
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CORREUTKDN  PEAK  VALUE 


Figure  29.  Correlation  vs.  Range  for  misregistration  in  X  direction 
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Figure  30.  Correlation  vs.  Range  for  misregistration  in  Y  direction 
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5.7  SAMPLING  USING  SQUARE  DETECTORS 


The  basic  FLIR  system  simulated  was  the  same  as  in  previous  experiments  except  for  the 
simultaneous  variation  of  the  sampling  rate  and  detector  size  in  both  directions.  Figure  31  shows 
the  relative  sizes  of  the  square  detectors  and  the  sampling  grid  for  the  cases  simulated  in  the 
experiment.  Table  6  describes  detector  dimensions  and  sample  spacings  for  both  square  and 
rectangular  detectors.  As  can  be  seen  from  Figure  32,  the  improvement  in  correlation  in  going 
from  1  to  1.5  samples  per  dwell  (spd)  is  laiger  than  the  improvement  in  going  from  1.5  to  2  spd. 


1  SAMPLE/IFOV 

det.  4 

«  •  •  • 


1.5  SAMPLES/IFOV 


2  SAMPLES/IFOV 


:  :  •  •  • 

•  •  • 

♦  4 

1  V! 

det.  1 


•  •  •  • 

det.  5 

•  •  •  • 


•  •  •  • 


•  •  •  • 


Figure  31.  Relative  size  of  square  detectors  (shaded  squares)  and  square  sampling  grids(dots) 
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Table  6.  Detector  and  sampling  descriptions 


DETECTOR 

NUMBER 

DETECTOR 

SIZE 

DETECTOR 

SIZE 

#  SAMPLE 
PER  IFOV 

X  DIMENSION 

pM 

(pRads) 

pM 

(pRads) 

(pRads) 

1 

36.6 

36.6 

1.0 

(80) 

(80) 

1.5 

2.0 

2 

36.6 

54.84 

1.0 

(80) 

(120) 

1.5 

2.0 

3 

36.6 

73.12 

1.0 

(80) 

(160) 

1.5 

2.0 

4 

22.84 

22.84 

1.0 

(50) 

(50) 

1.5 

2.0 

5 

54.84 

54.84 

1.0 

(120) 

(120) 

1.5 

2.0 

SAMPLE  SAMPLE  SAMPLE 

SPACING  SPACING  DENSITY 

X  DIMENSION  Y  DIMENSION 
pM  pM 

(pRads)  (pRads)  (SAMPLES/MRAD^) 


36.6  36.6  156 

(80)  (80) 

24.4  24.4  351 

(54)  (54) 

18.3  18.3  625 

(40)  (40) 

36.6  54.84  104 

(80)  (120) 

24.4  36.56  231 

(54)  (80) 

18.3  27.42  417 

(40)  (60) 

36.6  73.12  78 

(80)  (160) 

24.4  48.8  171 

(54)  (108) 

18.3  36.6  312 

(40)  (80) 

22.84  22.84  400 

(50)  (50) 

15.23  15.23  900 

(33)  (33) 

11.42  11.42  1601 

(25)  (25) 

54.84  54.84  69 

(120)  (120) 

36.6  36.6  156 

(80)  (80) 

27.4  27.4  278 

(60)  (60) 
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Figure  32.  The  influence  of  sampling  rate  variation  using  square  detectors 


Although  correlation  peak  value  may  not  be  directly  proportional  to  perfoimance,  one  notices 
from  the  slope  of  the  lines  that  a  sampling  rate  of  1  spd  produces  the  same  correlation  value  as 
1.5  spd  at  1000  meters  greater  range.  It  is  assumed  that  keeping  the  same  correlation  peak  value 
for  two  system  designs  would  restill  in  equal  performance  of  an  algorithm.  In  going  from  1.5  spd 
to  2.0  spd  the  increase  in  correlation  value  is  not  as  great  as  the  increase  in  going  from  1.0  to  1.5. 
The  correlation  values  for  1.5  spd  are  equal  to  those  for  2.0  spd  at  approximately  500  meters 
greater  range.  These  trends  hold  for  the  rectangular  detector  aspect  ratios  to  be  discussed  in  the 
next  section.  One  would  expect  that  for  a  system  design  in  which  the  optics  was  more  limiting, 
the  influence  of  the  detector  height  and  sampling  rate  would  not  be  as  great. 

5.8  SAMPLING  AND  DETECTOR  ASPECT  RATIO 

It  is  thought  by  some  researchers  that  the  image  quality  provided  by  rectangular  detectors  is  good 
enough  for  automatic  target  recognition  applications.  They  point  out  the  advantages  of  the 
improved  SNR  for  the  larger  detectors  as  well  as  easier  fabrication  of  the  array.  The  resulting 
taller  FOV  is  seen  as  an  advantage.  In  this  experiment  we  investigated  the  influence  of  detector 
height  alone  on  correlation.  No  atmosphere  was  used  and  no  noise  was  simulated.  The  detector 
height  was  changed  so  that  its  aspect  ratio  was  1 : 1  (x:y)  for  detector  #1 , 2:3  for  detector  #2,  and 
1:2  for  detector  #3. 

The  sampling  rate  was  varied  simultaneously  in  the  x  and  y  directions  to  be  1, 1.5  and  2  samples 
per  detector  subtense.  Thus,  if  the  detector  height  was  increased  for  a  particular  sampling  rate, 
the  physical  sample  spacing  increased,  even  though  the  sampling  rate  expressed  as  samples  per 
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detector  subtense  remained  constant.  The  increased  presample  filtering  along  with  the  increased 
sample  spacing  both  contribute  to  the  decreased  resolution.  Figure  33  iUustrates  the  relative  size 
of  the  detectors  and  the  corresponding  sampling  grids. 


1  SAMPLE/IFOV 
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det.  1 


1.5  SAMPLES/IFOV 


•  •  •  • 


2  SAMPLES/IFOV 
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•  • 
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Figure  33.  Relative  size  of  detectors  (shaded)  and  sampling  grid  (dots)  for  rectangular  detectors 
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The  curves  labeled  1, 4  and  7  in  Figure  34  are  the  same  as  curves  2, 5,  and  8  in  Figure  32  and  are 
shown  for  comparison  purposes.  Curves  2, 5,  and  8  and  curves  3, 6,  and  9  in  Figure  32  show  the 
results  for  detector  2  (aspect  ratio  2:3)  and  detector  3  (aspect  ratio  1:2)  respectively.  As  one 
would  expect,  when  the  detector  height  increased  (as  the  amount  of  prefiltering  increased),  the 
performance  decreased.  Also  as  would  be  expected,  increasing  the  sampling  rate  improved 
performance  in  all  cases. 
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RANGE  (m) 

Figure  34.  The  influence  of  detector  height  and  sample  spacing 


The  trends  for  square  detectors  and  rectangular  detectors  are  similar.  In  both  cases  increasing  the 
sampling  rate  increases  the  correlation.  Any  increase  in  the  presample  filtering,  whether  due  to 
taller  detectors  or  larger  square  detectors,  will  decrease  performance.  The  correlation  seems  to  be 
a  strong  function  of  sample  density,  almost  independent  of  the  number  of  samples  per  detector 
subtense.  For  evidence  of  this,  first  compare  curves  2  and  6  in  Figure  32  where  the  sample 
spacing  is  exactly  the  same.  Curve  2  corresponds  to  1  spd  while  curve  6  corresponds  to  1.5. 
Curve  6  is  slightly  lower  since  the  detector  is  larger,  even  though  the  number  of  spd  is  larger.  As 
a  second  example  look  at  curves  1, 5  and  9  in  Hgure  34,  all  of  which  correspond  to  the  same 
sample  spacing  in  the  cross  scan  direction  (see  Figure  33).  Again  the  curves  are  very  close 
together. 

If  we  look  at  the  relation  between  sample  density  (in  samples  per  mrad),  forgetting  about  the 
sample  rate  (spd)  we  see  in  Figure  35  that  correlation  is  a  strong  function  of  sample  density,  at 
least  up  to  around  500  pixels  per  square  mrad  where  the  data  starts  to  level  off.  All  three 
sampling  rates  have  been  mixed  in  the  graph.  The  data  presented  correspond  to  the  3  km  case 
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only.  The  data  for  4.5  and  6  km  arc  similar  and  lie  slightly  lower  in  conelation  than  the  values 
plotted  for  3  km.  This  is  consistent  with  our  experience  with  algorithm  performance  being  very 
dependent  upon  munber  of  pixels  on  taiget 
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Figure  35.  Correlation  peak  values  for  various  pixel  densities  for  targets  at  3  km 


5.9  SAMPLE-SCENE  PHASE 

This  experiment  examines  the  influence  of  the  alignment  of  the  sampling  grid  with  the  scene. 

The  rclative  alignment  of  the  sampling  grid  and  the  taiget  has  been  teraied  sample-scene  phase.*^ 
For  an  undersampled  system  the  appearance  of  the  taiget  is  dependent  upon  the  sample-scene 
phase.  The  effect  will  not  be  very  apparcnt  at  shorter  ranges  where  there  arc  several  pixels  on 
every  feature  of  the  taiget.  However,  at  longer  ranges  where  a  feature  is  one  sample  wide, 
different  alignments  of  the  feature  with  the  sampling  grid  can  seriously  distort  the  feature. 

For  systems  which  arc  sampled  at  two  samples  per  input  bandwidth,  (not  two  samples  per 
detector  subtense  unless  the  detector  is  the  component  which  limits  the  bandwidth)  and  which  arc 
ideidly  reconstructed  (something  which  never  happens  in  present  FLIRs,  even  in  second 
generation  systems),  there  should  be  no  variation  of  the  image  vrith  sample-scene  phase.  Less 
than  ideal  reconstruction  can  result  in  some  sample-scene  phasing  effects  even  for  a  two  sample 
per  dwell  system.  Typical  reconstruction  for  current  FLIR  systems  is  a  sample  and  hold  (far  from 
ideal)  or  a  linear  interpolation  (better,  but  not  as  good  as  could  be  done)  in  combination  with  a 
Gaussian  display  spot.  (If  the  display  is  flat  panel,  the  reconstruction  due  to  the  display  is  more 
Uke  a  rcct  function  or  sample  and  hold.)  For  ATRs  the  signal  is  not  reconstructed  but  is  fed 
directly  to  the  signal  processor,  and  therefore,  if  the  FLIR  has  no  antialiasing  filter,  even  a  system 
with  two  samples  per  dwell  could  exhibit  some  sample-scene  phasing  effects. 
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Simulation  of  small  changes  in  the  line  of  sight  of  the  FLIR  was  accomplished  by  starting  the 
sampling  at  different  points  in  the  reference  image.  Nine  different  sample-scene  phases  were 
tested  for  three  different  sampling  rates  and  three  different  ranges,  and  the  results  are  plotted  in 
Figure  36.  Data  points  of  the  same  shape  indicate  different  sample  scene  phases  for  the  same 
sampling  rate.  The  magnitude  of  the  spread  in  correlation  values  for  a  particular  sampling  rate  is 
an  indication  of  how  different  a  single  target  appears  simply  due  to  phasing. 
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Figure  36.  The  influence  of  sample  scene  phasing  for  three  different  sampling  rates 
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For  a  target  recognizer  which  is  trying  to  discriminate  between  several  different  targets,  a  small 
spread  of  correlation  values,  or  ideally  no  spread,  is  desired.  The  larger  the  spread,  the  easier  it 
would  be  for  one  target  to  appear  like  (have  a  high  correlation  with)  another  target.  Thus,  the 
spread  is  a  measure  of  robustness  of  the  system. 

As  one  would  expect,  any  factor  which  increases  the  size  of  the  sampling  grid  relative  to  the 
target  will  increase  the  spread  of  correlation  values  due  to  different  phases.  Decreasing  the 
sampling  rate,  while  holding  detector  size  constant,  will  increase  the  sampling  grid  spacing. 
Increasing  the  detector  size,  even  while  keeping  a  constant  sampling  rate,  wiU  increase  the 
sampling  grid  spacing.  Increasing  the  range  to  the  target  will  increase  the  size  of  the  sampling 
grid  when  projected  onto  the  target. 

Figure  36  shows  the  increasing  spread  of  correlation  values  with  decreased  sampling  rate  and 
increased  range.  It  is  seen  that  at  long  range,  the  lower  sampling  rate  results  in  a  very  large  spread 
in  correlation  values. 
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5.10  NOISE  AND  DETECTOR  SIZE 


Figures  37  through  39  show  the  perfonnance  versus  range  for  different  combinations  of  detector 
size  and  SNR  for  no  atmosphere,  “good”  atmosphere  and  “poor”  atmosphere.  For  the  case  of  no 
atmosphere,  Figure  37,  the  signal  in  the  image  is  very  large  compared  with  the  noise,  even  at  long 
range.  Comparing  the  results  for  three  cases  (1)  a  small  detector  (detector  4)  with  high  noise  level 
(0  =  4  quantization  levels)  (2)  a  medium  sized  detector  (detector  1)  with  a  medium  noise  level 
(o=  2.5)  and  (3)  a  large  detector  (detector  5)  with  a  low  noise  level  (o  =  1.66),  one  notices  that 
for  the  case  of  no  atmosphere,  the  degradation  due  to  lower  resolution  of  larger  detectors 
dominates  any  degradation  due  to  more  noise  of  the  larger  detectors.  Thrs  is  the  case  at  all  ranges. 
Correlation  is  also  plotted  for  detectors  1  and  5,  each  with  the  same  amount  of  noise  as  is  used 
with  detector  4.  This  is  done  to  get  a  feeling  for  the  magnitude  of  benefit  due  only  to  the  SNR 
improvement  of  the  larger  detectors. 

For  the  good  atmosphere  case.  Figure  38,  at  a  range  of  3000  meters  the  signal  in  the  image  is  still 
much  larger  than  the  simulated  noise  levels  and  the  degradation  due  to  the  poorer  resolution  of 
the  larger  detectors  is  the  dominating  effect.  That  is,  the  larger  detectors  result  in  lower 
correlation.  At  4500  meters  the  signal  in  the  image  has  been  degraded  to  the  point  that  the 
simulated  noise  now  has  significant  degrading  effect.  The  SNR  benefits  of  going  to  a  larger 
detector  (detector  4  to  detector  1)  are  being  offset  by  the  resolution  penalty,  resulting  in  the  data 
points  for  the  two  cases  overlapping.  At  6000  meters  the  case  is  just  opposite  that  at  3000  meters. 
The  signal  in  the  image  is  small  enough  that  the  improved  SNR  of  the  larger  detectors  provides 
more  benefit  than  the  improved  resolution  of  the  smaller  detectors.  Thus,  at  6000  meters,  the 
larger  detector  performs  better  than  the  smaller  one.  Figure  39  shows  the  results  for  the  “bad” 
atmosphere  where  the  SNR  is  so  low  that  the  larger  detectors  show  better  performance. 

Whether  a  smaller  or  larger  detector  has  a  higher  correlation  value  will  depend  upon  the  dynamic 
range  of  the  input  scene,  the  atmospheric  attenuation,  and  several  system  design  details  including 
whether  one  can  achieve  BLIP.  As  more  importance  is  put  on  longer  range,  degraded  battlefield 
performance  requirements,  FLIR  system  designs  will  tend  to  favor  (BLIP)  systems  with  larger 
detectors.  If  the  dominant  noise  source  was  not  due  to  signal  fluctuations  the  benefits  of  larger 
detectors  could  disappear  altogether  and  the  conclusions  drawn  about  the  optimal  detector  array 
configuration  could  be  quite  different 
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Figure  37.  Correlation  vs.  range  for  different  size  BLIP  detectors:  no  atmosphere 
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Figure  38.  Correlation  vs.  range  for  different  size  BLIP  detectors:  “good”  atmosphere 
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Figure  39.  Correlation  vs.  range  for  different  size  BLIP  detectors:  “bad”  atmosphere 


5.11  RESTORATION 

A  complete  understanding  of  the  best  image  quality  achievable  using  a  particular  FPA 
architecture  can  be  achieved  only  after  an  investigation  of  the  final  image  quality  after 
restoration.  Restoration  could  significantly  improve  the  system  performance  for  an  ATR  and  for 
a  human.  The  improvement  in  performance  due  to  restoration  could  be  used  to  offset  the 
degradation  in  perfonnance  due  to  an  array  which  was  easier  to  build.  The  magnitudes  of 
performance  improvement  achievable  for  different  system  designs  ate  presently  being 
investigated  for  simulated  system  designs  and  will  soon  be  measured  for  real  FLIR  systems  under 
an  ILIR  effort. 

5.12  CROSS  CORRELATION 

Correlation  values  measured  in  the  previous  tests  are  thought  to  relate  to  the  performance  of 
future  algorithms;  however,  efforts  to  link  performance  directly  with  a  simple  performance 
measure  such  as  recognition  range  have  met  with  little  success.  One  such  effort  to  try  to  translate 
the  variation  of  a  focal  plane  design  parameter  into  range  will  be  described. 

It  was  thought  that  perhaps  one  could  determine  a  range  at  which  the  correlation  of  a  target  with 
the  degraded  image  of  that  target  would  be  the  same  magnitude  as  the  “cross  correlation”  of  that 
target  with  another  target  If  the  improvement  in  the  FLIR  design  then  resulted  in  separation  of 
correlation  and  cross  correlation  values,  one  would  have  to  go  to  longer  ranges  in  order  to  have 
correlation  and  cross  correlation  values  be  equal.  The  difference  in  the  two  ranges  would  be  a 
measure  of  the  value  of  a  particular  improvement  to  the  FUR  design. 
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In  practice  it  was  found  to  be  very  difficult  to  find  a  range  at  which  the  correlation  and  cross 
correlation  were  the  same.  The  two  targets  used  were  different  sizes  resulting  in  a  low  cross 
correlation  value.  In  order  to  carry  out  the  test  we  would  have  to  zoom  the  smaller  target  so  that  it 
was  approximately  the  same  size  as  the  reference  target  in  order  to  maximize  the  cross 
correlation. 
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Section  6  Interlace  Effects  on  Tracking  Algorithms 


A  second  generation  of  ATR  prototypes  has  been  developed  and  third  generation  systems  are 
being  proposed  and  built.  Trackers  were  added  to  second  generation  ATR’s  (an  importent 
distinction  from  first  generation  ATR’s)  and  will  be  fully  integrated  into  third  pneration  ATR’s. 
While  second  generation  ATR’s  were  proposed  for  stabilized  platforms,  the  third  generation 
ATR’s  will  be  expected  to  close  the  loop  on  these  stabilized  platforms  as  well  as  operate  in  more 
complex  modes  and  scenarios.  One  of  the  concerns  about  these  complex  systems  is  the  impact 
that  a  scanning  sensor  may  have  on  tracking  algorithm  performance  under  motion  of  the 
platform.  The  interplay  of  the  sensor,  platform  and  tracking  algorithms  can  best  be  predicted 
through  simulation.  However,  this  more  limited  analysis,  which  did  not  include  simulation,  does 
provide  some  insight  into  the  negative  impact  of  interlace  misregistration  on  second  and,  by 
implication,  third  generation  ATR  systems. 

One  of  the  motion  disturbances  considered  here  is  jitter.  Jitter  may  be  defined,  for  our  purposes, 
as  random  wandering  of  the  LOS  about  the  point  of  interest.  The  main  component  of  jitter  is 
considered  to  be  the  residual  motion  remaining  after  stabilization  of  a  gimbaled  platform  to  a 
dominant  disturbance  such  as  the  beating  of  the  rotors  of  a  helicopter  or  the  vibration  due  to  a 
tank’s  tread,  etc.  These  “steady  state’’  disturbances  may  not  be  the  only  sources  of  jitter,  however. 
Sharp  impulses  from  carmon  and  trains  of  impulses  from  bursts  of  machine  guns  may  also  be 
present.  These  disturbances  may  be  important,  depending  on  the  scenario  in  which  an  ATR/ 
tracker  is  used.  The  jitter  from  these  disturbances  will  exceed  the  stabilization  specifications 
achieved  for  systems  designed  to  suppress  only  the  “steady  state’’  disturbances.*®  Therefore,  a 
worst  case  of  two  to  three  pixels  of  jitter  will  be  assumed  for  a  stabilized,  high  resolution, 
infirared  sensor  which  is  defined  here  as  2  degree  FOV,  1000  x  1000  pixels. 

Another  motion  disturbance  that  is  considered  here  is  that  of  panning  for  the  purposes  of 
centering  a  target  in  the  FOV.  Centering  a  target  is  a  closed  loop  process  that  first  accelerates  the 
center  of  the  field  of  view  (CFOV)  toward  the  target  and  then  decelerates  the  CFOV  so  as  to 
prevent  the  CFOV  from  overshooting  the  target.  The  accelerations  and  decelerations  are  on  the 
order  of  8.5  pixels/frame2.  The  peak  velocity  of  the  platform  during  the  centering  process  is 
approximately  50  pixels/frame. 

1\vo  types  of  scarming  sensors  are  considered  with  respect  to  the  above  distorting  motions, 
bidirectional  scanned  and  interlaced  (BDSI)  and  uni-directional  scanned  and  interlaced  sensor 
systems  (UDSI),  as  shown  in  Figure  2  on  page  5.  The  BDSI  sensor  system,  shown  in  Figure  40, 
displays  a  non-uniform  distortion  across  the  FOV,  under  a  umfoim  motion,  because  the  time 
separation  between  the  corresponding  pixels  of  alternate  fields  varies  from  virtually  zero  on  the 
right  hand  side  of  the  screen  to  approximately  a  fiame  time  on  the  left  hand  side  of  the  screen.  It 
is  assumed  that  the  systems  ate  designed  so  that  a  frame  starts  scanning  on  the  left  side  of  the 
FOV.  Rgure  41  shows  the  distortions  of  a  UDSI  sensor  system  under  uniform  motion.  The  time 
between  corresponding  pixels  of  the  alternate  fields  of  a  this  system  is  a  field  time  and  is  constant 
across  the  FOV.  The  distortion  under  this  motion  would  therefore  be  constant  across  the  FOV  for 
the  UDSI  sensor. 
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Figure  40.  Target  distortion  of  bi-directional  scanning  and 
interlaced  sensor  (BDSI)  caused  by  platform  motion 


Figure  41.  Target  distortion  of  uni-directional  scanning  and 
interlaced  sensor  (UDSI)  caused  by  platform  motion 
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The  basic  types  of  trackers  under  consideration  are  the  centroid  tracker  (CT),  the  correlation 
tracker  (COR)  and  the  predictive  tracker  (PT).  The  CT  determines  the  center  and  the  extent  of  a 
target  in  two  dimensions.  The  CT  employs  a  track  gate  which  is  larger  than  the  target  in  order  to 
provide  an  adequate  margin  for  target  and  platfonn  dynamics  and  errors  of  placement.  The  COR 
finds  the  relative  offset  for  maximum  correlation  between  a  new  target  image  (or  derivative  of 
that  image)  and  a  reference  image  (or  derivative).  The  COR  is  limited  by  its  offset  range  which  is 
defined  to  be  the  maximum  radius  (in  pixels)  over  which  the  full  correlation  of  the  two  images  is 
done  (at  each  point).  The  limiting  factor  for  the  offset  range  is  the  throughput  limitations  of  the 
COR,  which  increases  dramatically  with  offset  range.'®  The  PT  employs  a  CT  or  COR  to 
measure  the  positions  or  relative  positions  of  a  target  and  then  filters  these  measurements  and 
makes  a  prediction  for  the  next  position.  AU  of  the  above  trackers  may  utilize  either  a  single 
fiume  or  single  field  of  image  data  to  perform  their  processing  functions  and  will  be  referred  to 
as  either  frame  or  field  trackers. 

The  CT  will  fail  if  the  gates  are  too  small  or  badly  placed  so  that  a  portion  of  the  target 
consistently  lies  outside  the  target  gates.  The  gates  quickly  shrink,  losing  the  target  completely. 
This  may  happen  if  there  is  not  enough  margin  between  the  target  and  target  gate  to 
accommodate  the  dynamics  of  either  the  target  or  platform.  The  COR  tracker  may  fail  if  the 
target  signature  varies  quickly  (due  to  obscuration  etc.)  thereby  reducing  and  broadening 
correlation  result.  The  COR  can  also  faU  if  the  offset  range  for  correlation  is  exceeded  by  the 
dynamics  of  the  platform  or  target.  A  PT  will  fail  if  its  internal  CT  or  COR  fails  or  if  its 
prediction  is  grossly  in  error. 

6.1  EFFECTS  OF  JITTER 

Jitter,  if  it  is  expressed  as  frame  to  frame  offsets,  increases  the  throughput  requirements  for  the 
CT,  COR  and  PT.  The  CT  throughput  requirements  are  increased  by  the  expansion  in  the 
dimensions  of  the  gate  required  to  maintain  adequate  margia  The  COR  throughput  is  increased 
by  the  necessity  to  expand  the  offset  range  for  correlation  by  the  amount  of  jitter.  The  PT 
throughput  is  increased  as  the  incorporated  CT  or  COR  throughput  is  increased.  These  increases 
in  throughput  requirements  are  similar  for  a  BDSI,  UDSI  or  a  scanned  and  non-interlaced  sensor. 
The  non-interlaced  sensors,  however,  can  be  expected  to  work  as  well  with  jitter  as  without  since 
they  suffer  no  field  to  field  distortion. 

The  chief  impact  of  field  to  field  jitter  on  a  frame  CT  is  to  “bias”  the  measurement  of  the 
centroid,  as  shown  in  Rgure  42.  The  impact  is  most  important  at  long  ranges  and  for  small 
targets  where  the  aimpoint  may  be  significantiy  disturbed  by  the  “bias”. 

A  ftame  COR  tracker  is  also  impacted  by  field  to  fidd  jitter.  These  effects  on  a  frame  COR 
tracker  may  be  computed  for  a  binary  area  correlator  for  a  three  meter  by  three  meter  target 
(17x17  pixel  front  view  of  tank  at  5Km).  If  there  is  three  pixels  of  jitter  in  the  horizontal  direction 
between  the  reference  frame  and  the  new  frame,  and  the  segmentor  removes  the  jagged  edges  (ax 
pixels  in  horizontal  direction)  of  the  distorted  target  image,  then  there  will  be  35%  loss  of 
condation  (187  vs.  289  pixels  squared  or  a  change  in  target  area  of  102  pixels  squared).  A  loss 
of  this  amount  of  correlation  will  typically  be  sufficient  to  place  a  tracker  in  coast  mode.  Even  if 
the  loss  of  correlation  does  not  cause  the  loss  of  track  or  coasting,  the  uncertainty  of  the 
measurement  of  offset  (relative  target  location)  is  directly  increased  by  the  amount  of  jitter,  i.e. 
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3  pixels.  This  is  true  because  the  smaller  resulting  binary  target  correlates  equally  well  within 
3  pixels  (horizontal).  A  frame  CX)R  tracker  employing  a  non-interlaced  sensor  does  not  suffer 
these  degradations. 

An  edge  correlator  shows  the  worst  performance  loss  due  to  field  to  field  jitter  distortions.  In 
Figure  42  edge  correlation  plots  are  shown  for  three  cases,  in  which  the  new  edge  image  has 
suffered  no  distortion,  a  one  pixel  distortion  in  the  horizontal  direction,  and  2x2  pixel  distortion 
in  both  dimensions.  The  degradation  of  the  correlation  value  is  obvious.  It  is  unlikely,  given 
normal  variation  about  the  edge  of  a  target  due  to  segmentation,  that  a  reliable  offset  could  be 
determined  from  the  correlation  result  of  the  most  severely  distorted  image.  This  degradation  of 
the  correlation  scales  up  for  any  square  target  and  therefore  is  range  independent 

If  a  FT  incorporated  a  COR  tracker,  then  its  predictions  could  be  expected  to  suffer  from  the 
firailties  of  the  incorporated  tracker.  If  the  uncertainties  of  the  incorporated  tracker  are  modeled 
properly  then  the  PT  may  be  able  to  improve  on  the  offset  uncertainty.  However,  the  response  to 
the  most  severe  degradation  of  Figure  42  would  be  to  make  no  prediction,  i.e.  coast,  and  possibly 
lose  track. 
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Figure  42.  Binary  edge  correlator  with  field  distortions 
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6.2  IMPACTS  OF  TARGET  CENTERING 

A  panning  motion  of  the  sensor,  exhibiting  constant  acceleration  of  8  pixels  per  frame  squared, 
severely  impacts  a  CT  frame  tracker.  The  change  of  target  image  length  from  frame  to  frame 
(Figure  42)  for  a  BDSI  sensor  (target  on  the  left  side  of  the  FOV)  is  directly  proportional  to 
acceleration,  i.e.  eight  pixels  per  frame.  This  distortion  averages  out  the  centroid  between  fields, 
i.  e.  four  pixels  ahead  of  the  first  field  and  four  pixels  behind  the  second  field.  This  centroid 
“bias”  is  progressive  with  each  frame  and  after  three  frames  will  be  located  between  the 
separated  target  field  images  of  the  front  view  of  a  tank  at  5km,  used  above.  If  the  centroid  is  the 
aimpoint  of  this  “target”,  then  accurate  fire  is  sure  to  miss  the  target.  It  is  also  unclear  that  both 
separated  images  will  remain  in  the  track  gate.  One  or  both  of  the  field  images  may  walk  out  of 
the  gate  before  it  can  properly  resize  itself.  There  is  a  50%  chance  of  losing  track  under  these 
circumstances.  If  this  loss  of  track  is  to  be  avoided,  the  margin  between  the  target  and  track  gate 
must  be  greatly  increased  to  handle  the  increased  distortion  of  the  taiget.  For  the  target  used 
above  (front  view  of  tank  at  5km)  the  throughput  requirements  for  the  flame  CT,  suffering 
horizontal  panning,  would  be  increased  by  a  factor  of  three  above  that  required  for  the  dynamics 
alone.  Again  these  severe  degradations  are  not  present  in  a  non-interlaced  target  image. 

It  may  be  argued  that  a  ATRAracker  does  not  have  to  teU  the  weapon  system  to  fire  during 
centering.  However,  this  is  scenario  dependent.  It  is  not  hard  to  imagine  an  air  to  air  scenario  in 
which  the  sensor  system  is  constantly  trying  to  center  the  target  during  engagement  To  be  unable 
to  fire  during  centering  would  be  a  severe  handicap. 

The  above  panning  motion  presents  a  severe  challenge  for  a  COR  frame  tracker  as  well.  An  ^ge 
correlator  would  fail  instantly.  An  area  COR  tracker,  employing  a  BDSI  sensor,  could  also  fail 
instantly.  Assuming  jagged  edges  were  removed  by  the  segmentor,  the  correlation  value  for  the 
target  image  used  above  located  on  the  left  side  of  the  FOV  would  drop  50%  in  one  frame  time. 
In  addition,  there  would  be  a  8  pixel  offset  imcertainty.  If  the  jagged  edges  of  the  image  were  not 
removed  by  the  segmentor,  the  peak  correlation  value  would  be  reduced  by  25%,  and  agam  the 
offset  uncertainty  would  be  increased  by  eight  pixels.  This  is  very  ^or  performance  and 
eventually  the  target  image  fields  may  completely  separate  (depending  on  the  aspect)  and  cause 
tracking  failure.  Additionally,  8  pixels  uncertainty  increases  the  throughput  requirements  by  a 
factor  of  6  (compared  to  jitter).  While  a  UDSI  sensor  produces  half  the  distortion  of  the  BDSI 
sensor,  that  is  not  much  improvement  and  will  only  extend  the  operatioii  for  a  few  additional 
frames.  The  PT  would  fail  if  the  incorporated  CT  and  COR  trackers  fail. 

In  general  then,  aU  tracking  algorithms  could  be  expected  to  fail  at  fax  range  during  a  target 
centering  mode.  This  is  true  for  both  BDSI  and  UDSI  sensors. 

6.2.1  Field  TVackers  and  Target  Centering 

Field  trackers  would  seem  to  be  an  ideal  solution  to  the  distortion  problems  noted  above.  They 
would  suffer  no  field  offset  distortion  like  the  frame  trackers  that  employ  interlaced  systems. 
While  utilizing  the  same  throughput,  they  operate  at  twice  the  speed,  thereby  reducing  djmamics 
by  half  However,  they  also  have  their  problems.  They  work  with  a  half  resolution  target  in  the 
vertical  dimension  since  only  a  field  of  data  is  used  for  tracking.  This  may  reduce  maximum 
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range  by  a  factor  of  two  if  the  scenario  requires  targets  to  be  tracked  at  extreme  ranges  or  if  small 
targets  need  to  be  tracked.  In  addition,  a  FT  for  a  system  employing  a  BDSI  sensor  will  produce 
grossly  incorrect  predictions  during  a  target  centering  mode  due  to  the  effective  non-uniform 
sampling  of  that  system.  A  discussion  of  this  follows. 

Assuming  a  very  large  gate,  no  throughput  limitations  and  zero  delay,  a  PT  filter  was  designed 
for  the  apparent  motion  induced  by  the  centering  function.  A  1/2  pixel  of  error  was  allowed  for 
the  measuring  error  of  the  incorporated  CT  or  COR,  and  no  jitter  was  assumed.  In  other  words, 
this  result  represents  the  best  that  an  in-raster  PT  can  do.  Figure  43  compares  the  predictive 
output  of  the  PT  with  the  calculated  input  position  of  the  target  image  for  both  a  UDSI  and  BDSI 
field  tracker,  four  curves  in  all.  The  result  for  the  UDSI  sensor  appears  as  one  line  indicating 
excellent  prediction.  The  result  for  the  BDSI  input  is  not  so  smooth.  Both  curves  show  the  target 
being  centered  after  30  fields  (15  frames).  The  square  wave  acceleration  and  velocity  are  shown 
for  reference. 
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Figure  43.  Predictive  field  tracker  response  to  target  centering  -  UDSI  &  BDSI  sensor  systems 


The  differences  between  the  calculated  and  filtered  results  are  shown  more  clearly  in  Figures  44 
and  45.  The  predictive  error  for  the  UDSI  system  is  less  than  5  pixels,  and  the  peaks  of  the  error 
occur  at  the  beginning  and  end  points  of  the  accelerations  and  decelerations.  The  peak  predictive 
error  for  the  BDSI  sensor  system  is  30  pixels,  and  this  error  alternates  sign  with  every  point  Put 
simply:  fire  large  predictive  error  shown  for  the  BDSI  system  is  the  result  of  the  PT  over¬ 
estimating  and  imderestimating  the  target  position  from  field  to  field.  The  poor  prediction 
performance  of  the  PT  employing  the  BDSI  sensor  precludes  the  utilization  of  any  field  PT 
for  this  type  of  system. 
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PIXELS  PIXELS 
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6 J  TRACKER  SUMMARY 

In  summary,  it  has  been  shown  that  aU  interlaced  sensor  systems  present  harmful  image 
distortions  to  frame  tracker  algorithms  during  the  sensor  motions  of  jitter  or  panning.  In  addition, 
lai^e  increases  of  throughput  are  required  for  even  marginal  performance.  Field  trackers,  though 
immune  to  most  of  the  distortion  effects,  will  suffer  possible  range  limitations  due  to  the  half 
image  height  utilized  by  these  trackers.  It  has  also  been  shown  that  a  field  PT  may  not  be 
employed  with  a  BDSI  sensor  system  if  target  centering  is  required.  On  the  other  hand,  a  non¬ 
interlaced  sensor  is  not  expected  to  present  significant  distortions  to  trackers.  Throughput 
requirements  will  be  increased  only  in  direct  proportion  to  the  increased  dynamics  of  the 
platform  due  to  jitter  and  panning. 

Therefore,  for  good  tracking  algorithm  performance,  the  non-interlaced  sensor  is  preferred  for 
frame  tracker  algorithms.  The  second  choice  for  frame  tracker  systems  would  be  the  UDSI  sensor 
system,  though  severe  problems  may  be  anticipated  during  panning  and  centering  or  for  highly 
dynamic  targets  such  as  missiles  or  jets.  For  field  tracker  systems,  a  UDSI  sensor  system  should 
be  employed,  though  a  range  limitation  may  be  suffered  for  this  system  due  to  the  half  height 
target  image  of  each  field.  The  BDSI  sensor  should  not  be  utilized  in  any  tracking  system  (field 
or  frame)  that  may  experience  significant  jitter  or  relative  motion  such  as  that  produced  by  sensor 
panning,  platform  motion,  or  highly  dynamic  targets. 
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Section  7  Conclusions 


The  influence  of  various  infrared  FPA  parameters  was  simulated,  and  the  correlation  between  the 
simulated  image  and  a  starting  reference  image  was  measured  as  an  indicator  of  the  performance 
of  future  target  recognition  algorithms.  It  has  been  shown  that  misregistration  of  fields  of  an 
interlaced  FLIR  degrades  algorithm  performance  significantly.  Various  amounts  of 
misregistration  impacted  the  performance  of  several  types  of  trackers,  sometimes  simply 
degrading  performance  and  other  times  completely  breaking  lock.  As  a  result  of  this  it  is 
recommended  that  imagery  for  ATRs  and  trackers  be  noninterlaced. 

The  magnitude  of  the  performance  improvement  resulting  from  a  particular  FPA  design  change 
can  be  compared  with  improvements  expected  from  other  design  changes  by  reviewing  the 
results  of  the  appropriate  experiments.  While  making  the  comparisons  it  should  always  be  kept  in 
mind  that  the  variables  are  being  treated  separately  due  to  the  difficulty  in  fonning  general 
relationships  between  variables  which  are  applicable  to  all  FPA  fabrication  methods,  and  in  a  real 
system  there  may  be  interdependencies  between  variables  which  are  not  reflected  in  this  testing. 

The  performance  of  the  FLIR  system  with  varying  amounts  of  noise  and  nonuniformity  was  a 
strong  function  of  the  final  SNR  in  the  image.  At  low  SNRs  (at  long  range  or  in  a  degraded 
atmosphere)  correlation  levels  were  reduced  substantially.  The  same  level  of  noise  or 
nonuniforaiity  in  low  SNR  conditions  resulted  in  a  large  spread  of  correlation  values,  a  condition 
which  would  destroy  the  robustness  of  an  ATR. 

Large  detectors  destroyed  the  system  resolution  and  thus  the  correlation  values.  However,  if  the 
system  was  assumed  to  be  BLIP  limited,  larger,  lower  noise  detectors  were  shown  to  have 
advantages  in  low  SNR  conditions.  Increased  sample  density  improved  system  performance  as 
would  be  expected.  Sampling  the  scene  after  prefiltering  (after  optics  and  detector  spatial 
response)  at  less  than  two  samples  per  input  bandwidth  resulted  in  sample-scene  phase  problems 
which  led  to  a  spread  of  correlation  values.  Less  than  ideal  reconstniction  also  contributed  to  the 
phasing  problem.  Again,  the  spread  of  correlation  values  for  a  particular  sampling  geometry 
related  to  system  robusmess.  The  spread  was  range  dependent,  being  very  large  for  long  ranges 
and  low  sampling  rates  indicating  a  lack  of  robusmess  of  the  system. 

The  posable  benefits  of  image  restoration  should  be  explored  further  and  will  be  in  an  ILIR 
effort. 
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Appendix 


A-l  SAMPLE  EXPERIMENT  FILE 


in  order  to  connect  the  various  ‘C  programs  created  to  simulate  FLIR  FPA  par^eter  changes,  a 
system  was  devised  to  pipe  the  desired  ‘C’  programs  together.  The  mam  tool  of  this  piping  was 
an  experiment  ffle  used  as  input  to  the  piping  system.  An  experiment  file  wm  wntten  for  each 
FPA  parameter  change,  or  combination  of  changes,  designated  as  worthy  of  investigation  in  the 
experiment  These  parameters  included:  samples/IFOV,  detector  size/shape,  sample  scene 
phasing,  atmosphere,  noise,  and  nonuniformity.  Each  experiment  file  fisted  all  n^saiy  C 
programs  and  their  required  inputs  in  order  of  their  usage  in  the  pipmg  system.  The  following  is 
an  example  of  an  experiment  file: 


..fimage/m60_fr.ref 

..fimage/flirll.par 

4500 


file  containing  ref  image  info 
file  containing  flir  parameters 
range  in  meters 


optics 

./image/fin60_fr.atf 

131.aif 


flag  for  optics  inputs 
input  image 
output  image 


sampling 
131.arf 
131150.arf 
1.5  1.5 
1 1 


flag  for  sampling  inputs 
input  image  (after  optics) 
output  image 

x  samplefifov,  y  sampleAfov 
#  xphase,  #  yphase 


noise 

131150.arf 

1311501.arf 

4 

00 

12 


flag  for  noise  inputs 
input  image 
output  image 
#  frames 

mean  &  sigma  for  mean 
mean  &  sigma  for  variance 


lescale  flag  rescale  inputs 

..Amage/fin60_fr.arf  reference  input  image 
1311501.arf  input  image  (after  sampling) 

131150101.arf  ouqmt  image 

433  347  sub_width,out  sub_height 

1  interpolation  function  (bicubic) 


imgshift  flag  for  imgdrift  inputs 

.7image/ftn60_fr.arf  reference  input  image 
131150101.aif  input  image  (after  rescale) 

256  240  reference  image  centroid 

12 12  *  shifts  X,  #  shifts  y 

131 150101  Ldat  output  data  file 
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Each  “C’  program  was  coded  to  incoiporate  the  piping  system  by  reading  its  name  and  neces¬ 
sary  inputs  from  the  experiment  file  during  the  execution  of  the  program.  Thus,  by  redirection, 
the  piping  system  enabled  programs  to  read  any  experiment  file,  execute  the  program’s  desired 
functions,  and  output  the  experiment  file  for  the  next  program  in  the  piping  system  to  read  as  its 
input  An  example  of  a  typical  command  line  for  an  ejq)eriment  file  would  be: 

optics  <  131 120101  l.exp  1  sampling  I  noise  I  rescale  I  imgshift 

where  optics  is  the  first  program  called,  131 120101  l.exp  is  the  experiment  file  name,  and 
sampling,  noise,  rescale,  and  imgshift  are  the  subsequent  programs  called  in  that  order. 

The  experiment  file  naming  system  was  coded  digitally  due  to  a  fourteen  place  file  name  limit  of 
the  Stardent  computer  which  was  used  to  run  most  of  the  experiment  The  first  ten  digits  in 
experiments  with  no  atmosphere  were  coded  in  a  specific  order  to  indicate  target  range,  optics, 
detector,  number  of  samples/IFOV,  sample  scene  phasing,  noise,  nonunifoimity,  rescaling,  and 
correlation.  File  names  of  experiments  with  atmosphere  contained  an  eleventh  digit  to  indicate 
the  level  of  atmosphere  added  to  the  image.  The  following  table  is  the  coding  chart  used  to  create 
each  experiment  file. 


1 

target 

& 

aspect 

2 

range 

(m) 

3 

optics 

focal  I. 

apert  d. 

4 

Xdetect. 

Ydetect. 

microns 

5 

s/d 

6 

phase 

7 

noise 

8 

nonuni 

9 

rescale 

10 

X-cor 

11 

atm 

1«m60fr 

1.3000 

1.0.457 

1s;36.6 

1.1.0 

0.none 

0 

0 

I.Park 

1.m60fr 

sigmal 

2*m60rf 

2.4000 

0.2032 

36.6 

2.2.0 

1.3x3 

1 

1 

2.m60rf 

sigma2 

3=bmpfr 

3.4500 

2i>36.6 

5.1.5 

2 

2 

3.bmpfr 

sigma3 

4=bmprf 

4.5000 

54.84 

3 

3 

4.bmprf 

8igma4 

5.6000 

3>^6.6 

4 

8igma5 

73.12  •ig'n 

4.22.84 

22.84 

5.54.84 

54.84 

As  the  processing  proceeded  from  one  program  to  another,  each  intermediate  image  produced 
was  coded  with  an  additional  digit  to  indicate  which  FUR  parameter  changes  and/or 
interferences  had  been  added  to  the  image.  For  example,  a  three  digit  image  name,  such  as 
1 1  iMf,  meant  fliat  a  reference  image  of  the  first  target  (mbOfront)  at  the  closest  simulated  range 
(3000m)  had  only  optics  added  to  the  image,  whereas  a  seven  digit  image  name,  such  as 
111151  l.arf,  indicated  that  the  same  target,  range,  and  optics-added  image  also  had  1.5 
samples/IFOV,  sample  scene  phasing,  and  level  1  noise  added  to  the  image.  This  coding  was 
necessary  in  order  to  label  ttie  intermediate  images  as  each  experiment  file  was  read  by 

programs  in  the  piping  system.  If  the  intermediate  images  produced  by  each  program 
were  not  labeled  correctly,  the  next  program  called  in  the  piping  system  would  not  operate  on  the 
correct  image,  and  thus  the  final  correlation  value  would  be  invalid.  Labeling  intermediate 
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images  also  decreased  computer  run  time  because  parameter  changes  in  later  stages  of  an 
experiment  file  could  be  applied  to  an  intermediate  image  with  the  required  parameters 
previously  created  by  another  experiment  file. 

A-2  UST  OF  ACRONYMS 

ADASIM  an  ADA  computer  language  simulation  of  the  Hughes  Multifimction  Target 
Acquisition  Processor 

AFMAC  Advanced  FLER  Measurements  and  Analysis  Concepts 

(a  Wright-Patterson  program) 

ATR  automatic  target  recognizer 

BDSI  I-directional  scanned  and  interlaced 

CFOV  center  field  of  view 

COR  correlation  tracker 

CT  centroid  tracker 

F  Number  -  speed  of  lens  -  length/diameter  of  optic  aperture 

FFT  fast  Fourier  transform 

FLIR  forward  looking  infiared  system 

FOV  field  of  view 

FPA  focal  plane  array 

IFOV  instantaneous  field  of  view  -  detector  angular  subtense 

ILIR  independent  lab  in-house  research 

IR  infrared 

IRFPA  infrared  focal  plane  array 

LOS  line  of  sight 

MTF  modulation  transfer  function 

net  noise  equivalent  temperature  (difference) 

PT  predictive  tracker 

SNR  signal  to  noise  ratio 

SPD  samples  per  dwell 

UDSI  nni -directional  scaimed  atrd  interlaced 
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